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Abstract. This paper studies a large population dynamic game involving nonlinear stochastic 
dynamical systems with agents of the following mixed types: (i) a major agent, and (ii) a population 
of N minor agents where N is very large. The major and minor (MM) agents are coupled via both: (i) 
their individual nonlinear stochastic dynamics, and (ii) their individual finite time horizon nonlinear 
cost functions. This problem is approached by the so-called e-Nash Mean Field Games (e-NMFG) 
theory. A distinct feature of the mixed agent MFG problem is that even asymptotically (as the 
population size N approaches infinity) the noise process of the major agent causes random fluctuation 
of the mean field behaviour of the minor agents. To deal with this, the overall asymptotic (JV — > 
oo) mean field game problem is decomposed into: (i) two non-standard stochastic optimal control 
problems with random coefficient processes which yield forward adapted stochastic best response 
control processes determined from the solution of (backward in time) stochastic Hamilton- Jacobi- 
Bellman (SHJB) equations, and (ii) two stochastic (coefficient) McKean-Vlasov (SMV) equations 
which characterize the state of the major agent and the measure determining the mean field behaviour 
of the minor agents, (i) and (ii) are coupled in the following way: the forward adapted stochastic best 
response control processes in (i) involve the state of the major agent and the distribution measure 
corresponding to the mean field behaviour of the minor agents in (ii) where these in turn depend 
upon the best response control processes themselves. By introducing density functions corresponding 
to the state distribution measures of the agents the SMV equations may be expressed in the form of 
stochastic (coefficient) Fokker-Planck-Kolmogorov (SFPK) equations. Existence and uniqueness of 
the solution to the Stochastic Mean Field (SMF) system (SHJB and SMV equations) is established 
by a fixed point argument in the Wasserstein space of random probability measures. In the case 
that minor agents are coupled to the major agent only through their cost functions, the ejv-Nash 
equilibrium property of the SMF best response control possess is shown for a finite N population 
system where ejv = 0(1/ V~N). 
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1. Introduction. An important class of games is that of dynamic games with a 
very large number of minor agents in which each agent interacts with the average (or 
so-called mean field) effect of other agents via couplings in their individual dynamics 
and costs. A minor agent is an agent which, asymptotically as the population size 
goes to infinity, has a negligible influence on the overall system while the overall 
population's effect on it is significant. Stochastic dynamic games with mean field 
couplings arise in fields such as wireless power control [19], consensus dynamics [43] . 
flocking [42] , synchronization of coupled nonlinear oscillators [51] , crowd dynamics [9] 
and economics [50] [T] [12] . 

For large population stochastic dynamic games with mean field couplings and no 
major agents, the e-Nash Mean Field Games (e-NMFG) (or Nash certainty equivalence 
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(NCE)) theory was originally developed as a decentralized methodology in a series of 
papers by Huang together with Caines and Malhame, see [TH1 [H] for the MF linear- 
quadratic-Gaussian (MF LQG) framework, and [22] [201 [8] for a general formulation of 
nonlinear McKean-Vlasov (MV) type MF stochastic control problems. For this class of 
game problems a closely related approach has been independently developed by Lasry 
and Lions [23 EH [H3 03] where the term Mean Field Games (MFG) was initially used. 
For models of many firm industry dynamics, Weintraub et. al. proposed the notion 
of oblivious equilibrium by use of mean field approximations [IH] ■ The MF LQG 
framework is extended to systems of agents with ergodic (long run average) costs in 
[31] . while Kolokoltsov et. al. extend the e-NMFG theory to general nonlinear Markov 
systems |25j . The reader is referred to the survey paper [5] for some of the research 
on MFG theory up to 2011. 

The central idea of the e-NMFG theory is to specify a certain equilibrium rela- 
tionship between the individual strategies and the mass effect (i.e., the overall effect 
of the population on a given agent) as the population size goes to infinity [21] . Specif- 
ically, in the equilibrium: (i) the individual strategy of each agent is a best response 
to the infinite population mass effect in the sense of a so-called e-Nash equilibrium, 
and (ii) the set of strategies collectively replicates the mass effect, this being a dy- 
namical game theoretic fixed point property. The defining property of the e-NMFG 
equilibrium with individual strategies {u° : 1 < i < N} requires that for any given 
e > 0, there exists N(e) such that for any population size N(e) < N, when any agent 
j, 1 < j < N, distinct from i employs u°, then agent i can benefit at most e by 
unilaterally deviating from his strategy u°, and this holds for all 1 < i < N. The 
estimates in jT9] [22l [2TJ show e = 0(1/ \N) while distinct estimates are obtained in 
the framework of [25] . 

A stochastic maximum principle for control problems of mean field type is studied 
in [2] where the state process is governed by a stochastic differential equation (SDE) 
in which the coefficients depend on the law of the SDE. The reader is referred to (6] [7] 
for the analysis of forward-backward stochastic differential equations (FBSDEs) of 
mean field type and their related partial differential equations (PDEs). 

Recently, Huang [18] introduced a large population LQG dynamic game model 
with mean field couplings which involves not only a large number of multi-class minor 
agents but also a major agent with a significant influence on minor agents (see jT5J UM 
136] for static cooperative games of agents with different influences or so-called "mixed 
agents"). Since all minor agents respond to the same major agent, the mean field 
behaviour of minor agents in each class is directly impacted by the major agent and 
hence is a random process [T8]. This is in contrast to the situation in the standard 
MFG models with only minor agents. A state-space augmentation approach for the 
approximation of the mean field behaviour of the minor agents is taken in order 
to Markovianize the problem and hence to obtain e-NMFG equilibrium strategies 
[18] . An extension of the model in [18] to the systems of agents with Markov jump 
parameters in their dynamics and random parameters in their cost functions is studied 
in [48] in a discrete-time setting. See also [23] for the extension of the model in [18] 
to the case of systems with egoistic and altruistic agents. 

The model of [18] with finite classes of minor agents is extended in [37] to the 
case of minor agents parameterized by an infinite set of dynamical parameters where 
the state augmentation trick cannot be applied to obtain a finite dimensional Markov 
model. Due to the LQ structure of the problem an appropriate representation for 
the mean field behaviour of the minor agents as a random process is assumed which 
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depends linearly on the random initial state and Brownian motion of the major agent. 
Appropriate approximation of the model by LQG control problems with random pa- 
rameters in the dynamics and costs yields non-Markovian forward adapted e-NMFG 
strategies resulting from backward stochastic differential equations (BSDEs) obtained 
by a stochastic maximum principle |37j . 

In this paper we extend the LQG model for major and minor (MM) agents [T5] to 
the case of a nonlinear stochastic dynamic games formulation of controlled McKean- 
Vlasov (MV) type [22]. Specifically, we consider a large population dynamic game 
involving nonlinear stochastic dynamical systems with agents of the following mixed 
types: (i) a major agent, and (ii) a population of N minor agents where N is very 
large. The MM agents are coupled via both: (i) their individual nonlinear stochas- 
tic dynamics, and (ii) their individual finite time horizon nonlinear cost functions. 
Applications of the major and minor formulation may be found in charging control 
of plug- in electric vehicles [5H [35] , social opinion models QJj] with a finite number 
of leaders, and power markets involving large consumers and large utilities together 
with many domestic consumers represented by smart meter agents and possibly large 
numbers of renewable energy based generators [24) . 

A distinctive feature of the mixed agent MFG problem is that even asymptoti- 
cally (as the population size N approaches infinity) the noise process of the major 
agent causes random fluctuation of the mean field behaviour of the minor agents 
[TBI |3"T] . To deal with this, the overall asymptotic (N — > oo) mean field game prob- 
lem is decomposed into: (i) two non-standard Stochastic Optimal Control Problems 
(SOCPs) with random coefficient processes which yield forward adapted stochastic 
best response control processes determined from the solution of (backward in time) 
stochastic Hamilton- Jacobi-Bellman (SHJB) equations, and (ii) two stochastic (coef- 
ficient) McKean-Vlasov (SMV) equations which characterize the state of the major 
agent and the measure determining the mean field behaviour of the minor agents, 
(i) and (ii) are coupled in the following way: the forward adapted stochastic best 
response control processes in (i) involve the state of the major agent and the distri- 
bution measure corresponding to the mean field behaviour of the minor agents in (ii) 
where these in turn depend upon the best response control processes themselves. 

Existence and uniqueness of the solution to the Stochastic Mean Field (SMF) 
system (SHJB and SMV equations) is established by a fixed point argument in the 
Wasserstein space of random probability measures. In the case that minor agents are 
coupled to the major agent only through their cost functions, the ejv-Nash equilibrium 
property of the SMF best response control possess is shown for a finite N population 
system where ejy — 0(1/\N). As a particular but important case, the results of 
Nguyen and Huang [37] for MM-SMF LQG systems with homogeneous population are 
retrieved in Appendix G in 39 . In addition, the results of this paper are illustrated 
with a major and minor agent version of a game model of the synchronization of 
coupled nonlinear oscillators [5TJ (see Appendix H in [39]). 

It is to be emphasized that the non-standard nature of the SOCPs in (i), which 
consists of the coupling through the SMV equations in (ii), arises from a distinct 
feature of the problem formulation. The source of this non-standard nature is the 
game structure whereby the minor agents are (through the Principle of Optimality) 
optimizing with respect to the future stochastic evolution of the major agent's state 
which is partly a result of that agent's future best response control actions. Not 
only this feature vanishes in the non-game theoretic setting of one controller with 
one cost function with respect to the trajectories of all the system components (in 
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the game situation called agents), but also in the infinite population limit of the 
standard e-NMFG models with no major agents. This is true for completely and 
partially observed SOCPs. The nonstandard feature of the SOCPs here give rise to 
the analysis of systems with (non necessarily Markovian) stochastic parameters. Here, 
as in [33 [53], the theory of BSDEs (see in particular [3 EH g5l [16] ) is used in the 
resulting stochastic dynamic game theory. More specifically, we utilize techniques 
from 45 which applies the Principle of Optimality to a stochastic nonlinear control 
problem with random coefficients; this leads to a formulation of a SHJB equation 
by use of (i) a semi-martingale representation for the corresponding stochastic value 
function, and (ii) the Ito-Kunita formula. An application of Peng results to portfolio- 
consumption optimization under habit formation in complete markets is studied in 

The organization of the paper is as follows. Section[2]is dedicated to the problem 
formulation. A preliminary nonlinear SOCP with random parameters is studied in 
Section [3~] Section 21 presents a MF convergence theorem. The SMF system of equa- 
tions of the MM agents is given in Section [SJ and the existence and uniqueness of its 
solution is established in Section [5] The e-Nash equilibrium property of the resulting 
SMF control laws is studied in Section [7] Finally, Section [8] concludes the paper. 

1.1. Notation and Terminology. The following notation will be used through- 
out the paper. Let R n denote the n-dimensional real Euclidean space with the stan- 
dard Euclidean norm | • | and the standard Euclidean inner product - The trans- 
pose of a vector (or matrix) x is denoted by x T . tr(A) denotes the trace of a square 
matrix A. Let R nxm be the Hilbert space consisting of all (n x m)-matrices with the 
inner product < A,B >:= ti(AB T ) and the norm |A| :=< A. A > J / 2 . The set of 
non- negative real numbers is denoted by R+. T G [0, oo) is reserved to denote the 
terminal time. The integer N is reserved to designate the population size of the minor 
agents. The superscript N for a process (such as state, control or cost function) is 
used to indicate the dependence on the population size N. We use the subscript 
for the major agent Ao and an integer valued subscript for an individual minor agent 
{Ai : 1 < i < N}. At time t > 0, (i) the states of agents Ao and Ai are respectively 
denoted by Zq (t) and z^(t), 1 < i < N, and (ii) for the system configuration of minor 
agents (z^ (t), • • • , z^(t)) the empirical distribution is defined as the normalized 
sum of Dirac's masses, i.e., Sf := (1/N) Yli=i K N (t) where <5(.) is the Dirac measure. 
C{S) is the set of continuous functions and C k (S) the set of fc-times continuously 
diffcrentiable functions on S. We let C([0, T]; R n ) be the Banach space of continuous 
functions / : [0,T] -> R™ with the norm ||/|| := sup 0<t<T \f(t)\. The symbol d t 
denotes the partial derivative with respect to variables t. We denote D x and D xx as 
the gradient and Hessian operators with respect to the variable x. These are respec- 
tively denoted by d x and d xx when applied to a function defined on a one-dimensional 
domain. Let (O, J 7 , {Tt}t>o, P) be a complete filtered probability space. E denotes 
the expectation. The conditional expectation with respect to the cr-field V is denoted 
by Ey. We note that we may not display the dependence of random variables or 
stochastic processes on the sample point u> € Q. For an Euclidean space H we denote 
by ig([0, T]; H) the space of all {Q t }t>o-adapted -ff- valued processes f(t, to) such that 
E^\f{t,^)\ 2 dt<^. 

2. Problem Formulation. We consider a dynamic game involving: (i) a major 
agent Aq, and (ii) a population of N minor agents {Ai : 1 < i < N} where N is 
very large. We assume homogenous minor agents although the modelling may be 
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generalized to the case of multi-class heterogeneous minor agents [551 HE] (see [10]). 

The dynamics of the agents are given by the following controlled ltd stochastic 
differential equations (SDEs) on (f2, J 7 , {J-t}t>o, P)'- 



(2.1) dz»(t) = ±£f [t,z?(t),uZ(t),z?(t)]dt 

3=1 

1 N 

+ ^12 a o[t,z^(t),zf(t)]dw (t), 2^(0) = «b(0), 0<t<T, 

3=1 

1 w 

(2.2) dzf (t) = ^£/[Mf (t),«r(t),^(t),zf (*)]* 

3=1 

1 * 

+ ^ E ^(0) = *(0), 1 < i < N, 

3=1 

with terminal time T € (0, oo) where (i) Zq : [0,T] — > W l is the state of the major 
agent Ao and zf : [0, T] — > R n is the state of the minor agent Ai] (ii) Uq : [0, T] — ¥ Uq 
and : [0, T] — >• /7 are respectively the control inputs of Ao and Ai; (iii) /o : [0, T] x 
R n xU x R™ -> R n , cr : [0, T] x R™ xlM R nxm , / : [0, T] x K n x [/ x R" x R n -> R n 
and cr : [0,T] x R" x E n x M n -> M" xm ; (iv) the set of initial states is given by 
{^(0) = Zj-(0) : < j < N}, and (v) the sequence {(wj(t)) t >o ■ < j < N} denotes 
N + 1 mutually independent standard Brownian motions in R m . We denote the 
filtration Ft as the c-field generated by the initial states and the Brownian motions 
up to time t, i.e., T t ■— a{zj(0),Wj(s) : < j < N, < s < t}. We also set 
= <j{wo(s) : < s < t}. These filtrations are augmented by all the P-null sets 



r«'c> 
t 

in J- . 

For < j < N, u N 3 := {u$, ■ ■ ■ , uf_ v uf +1 , ■ ■ ■ ,u%}. The objective of each 
agent is to minimize its finite time horizon nonlinear cost function given by 

rT N 



dt. 



(2.3) J ( y(u%;u N ):=E (fl/N)^L [tX(t),u»(f),z?{t)]] 

J ° 3=1 

(2.4) jf(uf;u^):=E ((1/iV) ^>[i, zf(i), uf (t), z^(t),zf (t)])dt, 



3=1 
T N 

'E 

3 = 1 

for 1 < i < N, where L : [0,T] x R" x U x R n ^ R+ and L( Zl ,u u z ,x) : [0,T] x 
R™ x U x 1™ x R n — > E + are the nonlinear cost-coupling functions of the major and 
minor agents. For < j < N, we indicate the dependence of Jj on , u^a and the 
population size N by J^ 1 ;u^j). 

Remark 2.1. Under suitable conditions, the results of this paper may be adapted 
to deal with cost- couplings of the form: 

N N 
L [t,z»(t),u»(t),z?(t),-J24W> L[t,z»(t),u»(t),z»(t),z?(t),-J24W> 

3=1 3=1 

in E3)-p^p (see Appendix G in fS^). 

We note that in the modelling (|2.ip - (|2.4l) the major agent Ao has a significant 
influence on minor agents while each minor agent has an asymptotically negligible 
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impact on other agents in a large N population system. The major and minor (MM) 
agents are coupled via both: (i) their individual nonlinear stochastic dynamics (|2.ip ~ 
(|2.2[) . and (ii) their individual finite time horizon nonlinear cost functions (|2.3I) - (|2.4[) . 

We note that the coupling terms may be written as functionals of the empirical 
distribution SK by the formula J Rn (f>(x)5f (dx) — {1/N) YliLi 0(^1 (*)) f° r a bounded 
continuous function cf> in K™. 

2.1. Assumptions. Let the empirical distribution of N minor agents' initial 
states be defined by F N (x) = (1/N) £\ =1 l{ E z z (o)<x}, where l{ EZi ( )< x } = 1 if 
Ezi(Q) < x, and l{.E Zi (o)<x} = otherwise. We enunciate the following assumptions: 
(Al) The initial states {zj(0) : < j < N} are .Fo-adapted random variables 
mutually independent and independent of all Brownian motions {(wj{t))t>o : < j < 
N}, and there exists a constant k independent of N such that sup 0<:)<A r £7|zj(0)| < 



(A2) {Fjv : N > 1} converges to a probability distribution F weakly, i.e., for 
any bounded and continuous function <fi on 1" we have limjv_ >00 J R „ <f>(x)dFiq(x) = 



(A3) Uo and U are compact metric spaces. 

(A4) The functions fo[t,x,u,y], ao[t,x,y], f[t,x,u,y,z] and <j[t,x,y,z] are con- 
tinuous and bounded with respect to all their parameters, and Lipschitz continuous 
in (x, y, z). In addition, their first order derivatives (w.r.t. x) are all uniformly con- 
tinuous and bounded with respect to all their parameters, and Lipschitz continuous 
in (y,z). 

(A5) fo[t, x, u, y] and f[t, x, u, y, z] are Lipschitz continuous in u. 

(A6) Lo[t, x, u, y] and L[t, x, u, y, z] are continuous and bounded with respect to 
all their parameters, and Lipschitz continuous in (x, y, z). In addition, their first order 
derivatives (w.r.t. x) are all uniformly continuous and bounded with respect to all 
their parameters, and Lipschitz continuous in (y,z). 

(A7) (Non-degeneracy Assumption) There exists a positive constant a such that 



3. A Preliminary Nonlinear Stochastic Optimal Control Problem with 
Random Coefficients. Let (W(t)) t >o and (B(t)) t >o be mutually independent stan- 
dard Brownian motions in R"\ with Tf' B := a{W(s),B(s) : s < t} and := 
^{^(s) : s < t} where both are augmented by all the P-null sets in T . 

We now consider the following "single agent" nonlinear stochastic optimal control 



problem (SOCP) on (ft, J", { F t }t>o, P)- 

(3.1) dz(t,u) = f[t,L),z,u\dt + a[t,L),z]dW(t) + s[t,oj,z]dB(t), < t < T, 



where the coefficients /, er, <r and L are random depending on lo £ il explicitly. In (|3.ip ~ 
(|3T2|) : (i) z : [0,T] x Q -> R n is the state of the agent with 7" W ' B -adapted random 
initial state z(0) such that -E|;z(0)| 2 < oo; (ii) u : [0, T] x — ► U is the control input 
where U is a compact metric space; (iii) the functions / : [0,T] x 51 x 1" x U — ► R ra , 
a, <; : [0, T] x fix E™ — ► JR Tlxm are ^^-adapted stochastic processes; (iv) the admissible 
control set U is taken as U := {u(-) e U : u(t) is adapted to u-field J-^' B and 
E \u(t)\ 2 dt < oo}. We introduce the following assumptions (see [45] ) . 



k < oo. 



/ Rn 4>{x)dF(x). 



a [t,x,y]ao[t,x,y]> al, a[t,x,y, z]a T (t,x,y, z) > al, V(t,x,y,z). 



(3.2) 
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(HI) f[t, x, u] and L[t, x, u] are a.s. continuous in (a;, u) for each i, a.s. continuous 
in t for each [x,u), /[t,0,0] € £^([0, T]; R") and L[t, 0,0] € ^([0, T]; K + ). In 
addition, they and all their first derivatives (w.r.t. x) are a.s. continuous and bounded. 

(H2) <r[t, x] and x] are a.s. continuous in x for each t, a.s. continuous in t 
for each x and a[t,0], s[t,0] € L^ t ([0, T]; R nXm ). In addition, they and all their first 
derivatives (w.r.t. x) are a.s. continuous and bounded. 

(H3) (Non-degeneracy Assumption) There exist non-negative constants a± and 
a.2 such that 

<r[t, ui, x}a T [t, w, x] > a\I, q[t, uj, x]s T (t, u), x) > a^/, a.s., V(t,w,x), 

where a% or ot2 (but not both) can be zero. 

The value function for the SOCP ([3"3 ]) - (pT2]) is defined by (see [45]) 



(3.3) 4>(t,x(t)) = inf Ejrw I L[s,ui, z(s),u(s)]ds, 

u&A * J t 

where x(t) is the initial condition for the process x(s) := z(s), t < s. We note that 
</>(i, x(£)) is an J-^-adapted process which is sample path continuous a.s. under the 
assumptions (H1)-(H2). We assume that there exists an optimal control law u° G U 
such that 

<j)(t, x(t)) = E^w J L[s,u>,x(s),u°(s,uj,x(s))]ds, 

where x(-) is the closed-loop solution when the control law u° is applied. By the 
Principle of Optimality, it can be shown that the process 

(3.4) Q(t) := (f>(t,x(t)) + [ L[s,uj,x(s),u°(s,x(s))]ds, 

Jo 

is an {.F t w }o<t<T-niartingale (see (4]). Next, by the martingale representation theo- 
rem (see Theorem 5.7, Chapter 1, [54]) there exists an J^-adapted process ip(-, x(-)J 
such that 

(3.5) C(*) =0(O,x(O)) +/ 4> T (s,x{s))dW(s), fe[0,T]. 

Jo 

From (]32])-(l33]) and the fact that (f>(T, x(T)) = 0, it follows that 

C(T)= f L[s,uj,x(s),u°(s,x(s))]ds = 0(O,x(O)) + / V T (s, x(s))dW(s), 
Jo Jo 

which gives 

(3.6) <f)(0,x(0))= L[s,u,x(s),u°(s,x(s))}ds- ip T (s,x(s))dW(s). 

Jo Jo 

Hence, combining (|3.4|) - (|3.6|) yields 

(3.7) (f>(t,x(t)) = I L[s,aj,x(s),u°(s,x(s))}ds - ij; T (s, x(s))dW(s) 

r T 

T(s,x(s))ds~ i{; T (s,x(s))dW(s), ie[0,T], 
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where </>(s,x(s)), r(s, x(s)) and ip[s, a;(s)) are .Fj^-adapted stochastic processes (see 
the assumed "semi- martingale representation" form (3.5) in [45] ) . 

Using the extended Ito-Kunita formula (see Appendix A in |39| ) and the Principle 
of Optimality, Peng [55] showed that since 4>(t, x) can be expressed in the semi- 
martingale form (|3.7p . and if 4>{t,x), ip(t,x), D x (f>(t,x), D xx (f)(t,x) and D x ip(x,t) 
are a.s. continuous in (x,t), then the pair (0(s, x), ip(s, a;)) satisfies the following 
backward in time stochastic Hamilton- Jacobi-Bellman (SHJB) equation: 

(3.8) — d(/)(t,uj,x) = H[t,u,x, D x cf>(t,LJ,x)] + (a[t,uj,x},D x ip(t,uj,x)) 

D 2 xx cj)(t,uj,x)) \dt- ip T (t,uj,x)dW(t 7 Lu) 7 c/>(T,x)=0, 

where (t, x) E [0,T] x R™, a[t,ui,x] := a[t, lu, x]a T [t, lu, a;] + <,[t, uj, x]s T [t, uj, x] , and 
the stochastic Hamiltonian H : [0, T] x O x R™ x R™ — > R is given by 

H[t,Lu,x,p] := inf {{f[t,ui,x,u],p) + L[t,uj,x,u}}. 

We note that the appearance of the term {a[t, lu, x], D x tp(t, ui, x)) in equation (13.81) 
corresponds to the Brownian motion W(-) in the extended Ito-Kunita formula (|A.1|I 
for the composition of .F^-adapted stochastic processes 4>(t, ui, x) and z(t, lu) given in 
()3.7|) and (|3.1|) . respectively. 

The solution to the backward in time SHJB equation (|3.8[) is a unique forward in 
time F^-adapted pair (0, V0(i, 2:) = (0(^, w , ^), ^(^ w ! z)) ( see [45j[54]). We omit the 
proof of the following theorem which closely resembles that of Theorem 4.1 in [45j . 

Theorem 3.1. Assume (H1)-(H.3) hold. Then the SHJB equation k3.8\) has a 
unique solution (<P(t,x),ip(t,x)) in (L 2 - Ft ( [0, T] ; R) , L 2 - Ft ( [0, T] ; R" 1 )) . □ 

The forward in time F^-adapted optimal control process of the SOCP (|3.1j) - (|3.2[) 
is given by (see [35J) 

(3.9) u°(t, lu, x) :— arg inf H u [t, lu, x, D x <f>(t, lu, x), u] 

= arg inf {{f[t,u),x,u],D x <j){t,uj,x)) + L[t,Lu,x,u]} . 

By a verification theorem approach, Peng |45] showed that if the unique solution 
{4>,^p){t,x) of the SHJB equation p. 81) satisfies: 



(i) for each t, {<t>,^){t, •) is a C 2 (R") map from R n into 



x 



(ii) for each x, (4>, ip)(t, x) and (D x (j>, D xx (j>, D x ip){t, x) are continuous F^-adapted 
stochastic processes, then cj)(x, t) coincides with the value function (|3.3[) of the SOCP 

4. Major and Minor Mean Field Convergence Theorem. We take a prob- 
abilistic approach to show a "decoupling effect" result such that a generic minor 
agent's statistical properties can effectively approximate the distribution produced by 
all minor agents as the number of minor agents N goes to infinity (this is motivated 
by the analysis in Section 1.1 of [47] and in Section 8.1 of [22])- Therefore, each minor 
agent's state will be an independent copy of a nonlinear McKean-Vlasov (MV) process 
as N approaches infinity. 

Let tpo{uj,t,x) : D, x [0,T] x R ->• U and ip(io,t,x) : D, x [0, T] x R -> U be two 
arbitrary -measurable stochastic processes for which we introduce the following 
assumption: 
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(H4) (po(w, t, x) and <p(u>,t,x) are Lipschitz continuous in x, and Lp (u>,t,0) G 
([0, T]; U ) and ip{u, t, 0) € L^ ([0, T]; £7). 

We assume that ipo(t,x) := ifo(uj,t,x) and ip(t,x) := ip(uj,t,x) are respectively 
used by the major and minor agents as their control laws in (|2.1[) and (|2.2[) (i.e., 
u o = ancl u i — V f° r 1 ^ i < -^0- Then we have the following closed- loop 
equations with random coefficients: 

1 N 

dB o(t) = jj E /<>[*, %(t), Ml ^(t)),zf(t)]dt 
i=i 

1 - 

+ jv£ ff o[t,^(t),^(t)]dt«o(t), ^(0) = «b(0), 0<i<T, 



1 w 

+ -2 < r[t,5f(t),^(i) J ff(i)]d«, i (i) J ff(0) = 2i (0), 1<*<J\T. 



JV ■ 
i=i 

Under (A4)-(A5) and (H4) there exists a unique solution (zq (•), • • • , z^(-)) to the 
above system (see Theorem 6.16, Chapter 1 of [S3], page 49). 
We now introduce the McKean-Vlasov (MV) SDE system 

dz {t) = fa[t,z (t),(pa(t,z (t)),iit]dt + a [t,z (t), (i t ]dw (t), < t < T, 
dz(t) = f[t, z(t), ip(t, z(t)),Ht, Ht]dt + a[t, z(t), i4,Ht]dw(t), 

with initial condition (zo(0), 1(0)), where for an arbitrary function g e C^IR 5 ) for 
appropriate s, and probability distributions /it and fi® in M™, we set 

g[t,z,nt]= g[t,z,x]fi t (dx), g[t, z, (i° , fi t ] = / z,x,y]fi°(dx)fi t (dy), 



when the indicated integrals converge. In the above MV system (zo(-), z (')i M?.) >/*(•)) 
is a "consistent solution" if (zo(-),z(-)) is a solution to the above SDE system, 
< t < T, is the conditional law of z(t) given Tf° (i.e., /i< := C(z{t)\T? a )), and /uj?, 
< t < T, is the unit mass measure concentrated at zo(t) (i.e., /ij = <5z (t)). 

Under (A4)-(A5) and (H4) it can be shown by a fixed point argument that there 
exists a unique solution (zo(-), z(-), fi9yfi^j to the above system (see Theorem 1.1 in 
[17] or Theorem 16 . 71 below) . 

We also introduce the equations 

dz (t) = f [t,zo{t),ifo(t,z (t)),fit]dt + a [t,zo(t),fi t ]dw (t), < t < T, 
dzi(t) = /[t,2i(t),^(t,f i (t)),/x°,// t ]cit + (T[t,2 i (t),/i°,// t ](iu; i (t), 1 < % < N, 

with initial conditions z~j(0) = Zj(0), < j < N, which can be viewed as N indepen- 
dent samples of the MV SDE system above. We develop a decoupling result below 
such that each , 1 < i < N, has the natural limit z% in the infinite population limit 
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(see Theorem 12 in The proof of the following theorem is given in Appendix B 

in [35]. 

Theorem 4.1. Assume (Al), (A3)-(Ab) and (H4J hold. Then we have 



(4.1) sup sup E\zf {£) - Zj{t)\ = 0(1/VJV), 

0<j<N 0<t<T 

where the right hand side may depend upon the terminal time T. □ 

5. The Mean Field Consistency Condition. In the formulation (I2.1[) - (|2.4p 

all minor agents are reacting to the same major agent and hence the major agent has 
non-negligible influence on the mean field behaviour of the minor agents. In other 
words, the noise process of the major agent Wq causes random fluctuation of the 
mean-field behaviour of the minor agents and makes it stochastic (see the discussion 
in Section 2 of [TB] for the major and minor (MM) linear-quadratic-Gaussian (LQG) 
model). 

5.1. The Major Agent' Stochastic Mean Field (SMF) System. We con- 
struct the stochastic mean field (SMF) system for the major agent in the following 
steps. 

Step I (Major Agent's Stochastic Hamilton- J acobi- Bellman (SHJB) Equation): 
By the decoupling result in Theorem 14.11 which indicates that a single minor agent's 
statistical properties can effectively approximate the empirical distribution produced 
by all minor agents, we may approximate the empirical distribution of minor agents 
<5q with a probability measure fx^.y Since the noise process of the major agent wq 
influences the mean field behaviour of minor agents, is a stochastic measure 
depending on wq. 

Let /Xi(w), < t < T, be a given exogenous stochastic process such that (j,o(dx) := 
dF(x) where F is defined in (A2). Then we define the following SOCP (|33]) - ([3T2]) 
with J 7 ™ -adapted random coefficients from the major agent's model (|2.1[) and ()2.3I) 
in the infinite population limit: 

(5.1) dz (t) = f [t, z {t),u (t), fj, t (w)]dt + a [t, z (t), fit(u})]dw (t,cj), z (0), 



(5.2) inf Jo(u ) := inf E / L [t, z (t), u (t), \i t (uj)]dt 

uo g Uo ua&Aa L Jq 

where we explicitly indicate the dependence of the random measure /U/.n on the sample 
point oj G SI. 

The value function of the major agent's SOCP (|5.1[) - (|5.2j) is defined by (see [45] ) 

(5.3) 0o (*,&(*)) = inf Ejr^o / L [s, z (s), u {s), fi s (uj)]ds, 

where x(t) is the initial condition for the process x(s) := z (s), t < s (see (|3.3p ). As 
in Section[3] tfio(t,x(t)) has the form (see (|3.7[l ) 

T i-T 

t I 



cf> (t,x(i)) = j r (s, x(s))ds- J 4% (s,x(s))dw (s), t£[0,T], 

where 4>o ( s : x ( s )) > ro (s,x(s)) and if>o(s, x(s)) are J-"™°-adapted stochastic processes. 
If (j>o(t,x), ipo(t,x), D x <f>o(t,x), D^ x <fio(t,x) and D x ipo(x,t) are a.s. continuous in 
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(x,t), then (see [35]) the pair (<f>o(s, x), tpo(s, x)] satisfies the following stochastic 
Hamilton- Jacobi-Bellman (SHJB) equation (see (|3.8[) ): 

(5.4) - d(j> {t,u),x) = H [t,L>,x,D x <f> (t,L),x)] + (a [t,x, fit(u>)], D x ip (t,u>,x)) 
+ 7; iT ( a o[ t ,u,x}Dl x <p Q (t,Lj,x)) dt-tpQ(t,u} ) x)dw (t,uj), ^ o (T,x) = 0, 

where (t,x) £ [0,T] xE", ao[t,uj, x] := <ro[t, x, ^(oj)}^ \t , x, /if(w)], and the stochastic 
Hamiltonian H :[0,T]x!lxM"x R n -> R is given by 

H Q [t,u,x,p] := inf {(/ [i, x, u, /i t (uj)},p) + L [*, Mt(w)]} . 



The solution to the backward in time SHJB equation (|5.4I) is a forward in time .F™ - 
adapted pair [<po(t,x),ipo{t,x)) = (<po(t, u),x),ipo(t,U!,x)\ (see [35]V 

We note that the appearance of the term (<Jo[t, x, nt(^)],D x ipo(t, ui, x)) in equa- 
tion (|5.4p corresponds to the major agent's Brownian motion Wq(-) in the extended 
Ito-Kunita formula (see (|A. 1 1) in [33]) for the composition of J 7 ™ -adapted processes 
<f>o(t,u!,x) and zo(t,u>) in (|5.ip . 

The best response control process of the major agent's SOCP (|5.ip - (|5.2p is given 
by (see [35] ) 

(5.5) ug(t,a;,x) = ug(i,x|{/i s (w)}o< s <r) := arg inf H% [t, w, x, u , D x <f> (t, lo, x)] 

UqGUo 

= arg inf {(f [t,x,u ,fj, t (u})],D x <p (t,u},x)) + L [t,x,u ,nt(v)]}, 
u eu 

where the infimum exists a.s. here and in all analogous infimizations in the paper 
due to the continuity of all functions appearing in Hq° and the compactness of Uq. It 
should be noted that the stochastic best response (SBR) control Uq is a forward in time 

-adapted process which depends on the Brownian motion wo via the stochastic 
measure < t < T. The notation in (|5.5p indicates that Uq at time t depends 

upon the stochastic measure fi s (u>) on the whole interval < s < T. 

Step II (Major Agent 's Stochastic Coefficient McKean- Vlasov ( SMV) and Stochas- 
tic Fokker- Planck- Kolmogorov ( SFPK) Equations) : By substituting the best response 
control process Uq (|5.5p into the major agent's dynamics (|5.ip we get the following 
stochastic McKean- Vlasov (SMV) dynamics with random coefficients: 

(5.6) dzo(t, u) = f [t, Zq, u%(t, u, z%), Ht{u)]dt + a [t, z%, n t (u))]dw (t, u), 

with Zq(0) = zq(0), where /o and Co are random processes via the stochastic measure 
\i and Uq. The random measure of the major agent fi® (to), < t < T, is denoted as 
the unit mass measure concentrated at Zo(i,w) (i.e., ${oj) = <^g({, w ))- 

An equivalent method to characterize the SMV of the major agent is to express 
(|5.6p in the form of stochastic Fokker-Planck-Kolmogorov (SFPK) equation with ran- 
dom coefficients: 



(5.7) dp° s (t,uj,x) = ( - (D x , fo[t,x,Uo(t,u},x),fXt(u})]p°(t,LJ,x)) 
+ -^r{Dl x ,a Q [t,w,x]p° s {t,w,x)))dt 

- (D x ,o- [t,x,iit(u)]p s (t,u),x)dw {t,u})), p° s (s,u,x) = S z o (s Lo) (dx), 
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where < s < t < T. p°(t, lu, x) is the conditional probability density of z<$(t, uS) given 
J 7 ™ and has the initial condition p®(s,u>,x) = o~ z °( s ,u)(dx). 

We now summarize the major agent's stochastic mean field (SMF) system: 



(5.8) - d(f> (t,u,x) = H [t,u,x,D x <f> (t,u,x)] 

dt 



+ (a [t,x, /j, t (iv)],D x ij) (t,u),x)) + -tr(a [t,u,x]Dl x (t> (t,u,x)) 



- i>l (t, lo, x)dw (t, w), <po(T, x) = 0, [MF-SHJB] 

(5.9) ug(i,w,aj)=«g(t,j;|{/i.(w)}o<»<r) [MF-SBR] 

:= arg inf {(fo[t, x, u , Ht(u)], Ac<Ao(*> u, x)) + L [t, x, u , ^t(w)] }, 
u eu 

(5.10) dz^ui) = f [t,Z°,U°(t,LO,Z°),H t (Lo)]dt 

+ <T [t,z° ,m{u)]dw {t,oj), z°(0) = z (0), [MF-SMV] 

where (t,x) e [0,T] x W\ The solution of the system (HT8l) - (|5.10l) consists of 4-tuple 
J-^° -adapted random processes (4>o(t, oj, x),ipo(t, oj, x), u^(t, w, x), 2g(t, oj)) , for a given 
exogenous stochastic process fit(oj), where (i® (oj) is to be interpreted as the unit mass 
measure concentrated at the major agent's state z$(t,oj) (i.e., $ (oj) = <J z g(t lU) )). Note 
that /xt(w) will be characterized via the mean field consistency condition as the random 
measure of minor agents' mean field behaviour. 

5.2. The Minor Agents' Stochastic Mean Field (SMF) System. We 

construct the stochastic mean field (SMF) system for a "generic" minor agent i, 
1 < i < N, in the following steps. 

Step I (Minor Agent's Stochastic Hamilton- J acobi- Bellman (SHJB) Equation): As 
in Section IS~T1 let fi t , < t < T, be the exogenous stochastic process approximating 
the empirical distribution produced by all minor agents in the infinite population limit 
such that fi (dx) = dF(x) where F is defined in (A2). We let (j$(oj), < t < T, be 
the unit mass measure concentrated at the major agent's state Zg(i, a;) obtained from 
the major agent's SMF system f5T5 ]) - ([5TTD]) . 

We define the following SOCP (|3.1[1 - (13.21) with J"™ -adapted random coefficients 
from the i th generic minor agent's model (12.21) . (|2.4p in the infinite population limit: 

(5.11) dzi(t) = f[t, Zi(t), Ui(t),fi°(uj), nt(uj)]dt + a[t, z<(t), ^(w), iJ,t(oj)]dWi(t,oj'), 

(5.12) inf Ji(ui) := inf e\ [ L[t, zt(t), «<(«), v$(u), A*t(w)]dtl , 24 (0), 

Ui£ht Ui&J L Jq 

where we explicitly indicate the dependence of the random measures /m)^ and on 
the sample point oj £ Q. 

The value function of the generic minor agent's SOCP (|5 . 1 (|5 . 12[) is defined by 
(see @5] ) 

(5.13) <j>At,x(t)) = inf E-pVQ \ L[s, Zi(s),Ui(s), (j%(oj), (i s (oj)]ds, 

Ui£U * J t 

where x(t) is the initial condition for the process x(s) := Zi(s), t < s (see (|3.3p ). As 
in Section[31 (f>i(t,x(t)) has the form (see (|3.7[1 ) 

cf>i(t,x(t)) = / Ti(s,x(s))ds - / ipf(s,x(s))dw (s), t £ [0,T], 
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where (j)i(s,x(sj), IMs, x(s)) and ipi(s,x(s)) are Ff° -adapted stochastic processes. 
If 4>i(t,x), ipi(t,x), D x <j)i{t,x) and D^ x (j)i(t, x) are a.s. continuous in (x,t), then the 
pair (4>i(s, x), ifji(s, x)J satisfies the following backward in time stochastic Hamilton- 
Jacobi-Bellman (SHJB) equation [45] (see (l3~H) V. 



(5.14) - d(f>i(t,u,x) = H[t,u,x,D x (f) l (t,w,x)] + ^tr(a[t,ui,x}Dl x (f> i (t,Lj,x)) 

- tpf(t,uj,x)dwo(t,uj), (f>i(T, x) = 0, 



dt 



where (t, x) G [0,T] x M™, a[i, := ^°(w), ^ t (w)]CT T [i, x, ^°(w), ^ t (cj)], and 

the stochastic Hamiltonian H : [0, T] x O x M" x M™ — > R is given by 

uj, x,p] := inf {</[*, x, u, $(u}), nt(u)],p) + L[t, x, u, $(uj), Ht{uj)]}. 



The solution to the backward in time SHJB equation (|5.14[) is a forward in time T™ - 
adapted pair (4>i(t, x), ipi(t, x)) = (<j)i(t,uj,x),'ip i (t,uj,x)) (see [IS]). We note that 
since the coefficients of the SOCP (|5.1ip - (|5.12p are ^"'"-adapted random processes 
we have the major agent's Brownian motion wq in f|5 . 14[) which allows us to seek for 
a forward in time adapted solution to the backward in time SHJB equation (|5.14l) . 

It is important to note that in (|5.14[) unlike the major agent's SHJB equation (I5.4p 
we do not have the term (cr[t, x, (to), /j,t{oj)]D x ipi(t, uj, a;)} since the coefficients in 
the minor agent's model (I5.11l) - (|5.12l) are J 7 ™ -adapted random processes depending 
upon the major agent's Brownian motion (wo) which is independent of the minor 
agent's Brownian motion (wi) (see the extended Ito-Kunita formula (jA.lj) in [39 ) . 

As in Section 15. 1\ the best response control process of the minor agent's SOCP 
(l5Tro - (l5TT2l is (see [45]) 

(5.15) 

u°(t,uj,x) ee u°(t,x\{n a s (uj), (i s (uj)}o <s <t) := arg inf H u [t, u, x, m, D x 4>i(t, uj, x)] 
= arg mf {(f[t, x, u, $(uj), ^t(w)], D x (f>i(t, uj, x)) + L[t, x, u, $(uj), fi t (u)]}, 

where the infimum exists a.s. here and in all analogous infimizations in the paper 
due to the continuity of all functions appearing in H u and the compactness of U. It 
should be noted that the stochastic best response (SBR) control of the generic minor 
agent u° is a forward in time J 7 ™ -adapted random process which depends on the 
Brownian motion wq via the stochastic measures ${uj) and Ht(uj), < t < T. The 
notation in (|5.15j) indicates that u° at time t depends upon the stochastic measures 
P? s (uj) and /i s (w) on the whole interval < s < T. 

Step II (Minor Agent's Stochastic Coefficient McKean- Vlasov (SMV) and Stochas- 
tic Fokker- Planck- Kolmog or ov ( SFPK) Equations) : By substituting the best response 
control process u° (|5.15|) into the minor agent's dynamics (|5.1ip wc get the following 
stochastic McKean- Vlasov (SMV) dynamics with random coefficients: 

(5.16) dz?(t,w,w') =f[t,z?,u° i (t,uj,z i ),fj, t (uj),f H (uj)}dt 

+ a[t,z°,f4(uj),fi t (uj)]dwi(t,uj'), z?(0) = Zi (0), 

where / and a are random processes via the stochastic measures /x° and \i, and the 
best response control process u° which all depend on the Brownian motion of the 
major agent (wq). 
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Based on the decoupling effect (see Section @] or the theory of propagation of 
chaos in [?7]) the generic agent's statistical properties can effectively approximate 
the empirical distribution produced by all minor agents in a large population system. 
Hence, we obtain a new stochastic measure /it(w) for the mean field behaviour of 
minor agents as the conditional law of the generic minor agent's process z°(t, to) given 
We characterize (k(w), < t < T, by P(z°(t,uj) < a\T™°) = f"^ ft(t, u, dx) 
a.s. for all a € M™ and < t < T, with jlo(dx) = /j,o(dx) — dF(x) where F is defined 
in (A2). 

An equivalent method to characterize the SMV of the generic minor agent is to 
express (|5.16[) in the form of stochastic Fokker-Planck-Kolmogorov (SFPK) equation 
with random coefficients: 

(5.17) dp(t,u),x) = ( - (D x , f[t,x,u°(t,u>,x), [x°(u>), fj, t (u)]p(t,u,x)) 

+ ^{Dlx^it^^}?^^^)))^, p(0,x) =p (x), 

in [0,T] x R™ where p(t,w,x) is the conditional probability density of z°(t, oS) given 
J-]" . By the decoupling effect (see Section |4} it is possible to characterize the mean 
field behaviour of minor agents in terms of generic agent's density function p(t, u, x). 
The reason that the generic minor agent's FPK equation (|5.17j) does not include the 
ltd integral term with respect to u)j is due to the fact that p(t, w, x) is the conditional 
probability density given , and the independence of the Brownian motions wq and 
Wi, 1 < i < N. 

The density function p(t, w, x) generates the random measure of the minor agent's 
mean field behaviour /^(w) such that fi,(t,uj,dx) = p(t, uj, x)dx (a.s.), < t < T. 

We note that the major agent's SOCP (pTT|) - ([5~^|) and minor agent's SOCP (|?TT|) - 
(|5.12p may be written with respect to the random density p(t,uj,x) of the stochastic 
measure /j,(t, ui, dx) by jj,(t, w, dx) = p(t, w, x)dx (a.s.), < t < T. 

The mean field games (MFG) or Nash certainty equivalence (NCE) consistency 
(see [22] and [30]) is now imposed by letting /tt(w) = Ht{oj) a.s., < t < T, or 
p(t,uj,x) — p(t,u,x) a.s. for (t,x) € [0, T] x R™. The MFG consistency is demon- 
strated in: (i) the major agent's stochastic mean field (SMF) system (|5.8p - (|5.10l) 
together with (ii) the following summary of the minor agents' SMF system: 



(5.18) — d(f>(t,u,x) = H[t,u,x, D x <fi(t,Lu,x)} + -tr (a[t, ui, x]D^ x (f)(t,uj, x)) 



dt 



-ip T {t,uj,x)dw (t,uj), <t>(T,x)=0, [MF-SHJB] 

(5.19) M °(t,w,x) = U o (t,a;|{^(w), Ms H} < s <T) [MF-SBR] 

= arg inf {(f[t, x, u, (i°{u), /m(uj)], D x c/)(t, u, x)) + L[t, x, u, ^°(uj), fi f (uj)}} , 

(5.20) dz°(t,uj,Lu') = f[t,z ,u (t,io,z°),fi t (io),fi t {uj)]dt 

+ a[t, z°, fj%(u), ^ t {io)]dw{t, uj 1 ), [MF-SMV] 

where (t,x) E [0,T] x R™, and z°(0) has the measure /J,o{dx) — dF(x) where F is 
defined in (A2). We note that in the minor agents' SMF system (l5TT8]) - ([530)) we 
dropped index i from the generic minor agent's equations (|5.11[) - (|5.16[) . 

Note that the major and minor (MM) agents' SMF systems ([5~8 ]) - ([5TT0j) and 
(|5.18p - ()5.20p are coupled together through the stochastic measures (oj) and /xt(w). 

The MM SMF system is given by the MM agents' coupled SMF systems (pT5)) - 
([BTTU]) and (j5~T5| - ([5^0l) . The solution of the MM SMF system consists of 8-tuple 
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J-^" - adapted random processes 

(<f>o(t,cj,x),ipo (t,w,x),Mo(i,w,x),2;o(t,a;),(?!)(t,a;,a;),V'(t,a;,a;),w (i,w,a;),z (t,a;)), 

where the random measure p% (uS) is the unit mass measure concentrated at the major 
agent's state Zg(i,u;), i.e., p®(u) = 5 z °(t,u))i and z°(t,u>) generates the conditional 
random law pt{u), i.e., P(z°(t,uj) < a\J-™°) — f_ pt{<^,dx) for all a £ R ra and 
< t < T. 

We note that the solution to the MM SMF system is a "stochastic mean field" 
in contrast to the deterministic mean field of the standard mean field game problems 
in [22] [20] HH1 [SSI 1SD] ■ If the noise process of the major agent vanishes then the MM 
SMF system reduces to a deterministic MF system (see (6)-(9) in [20 ). 

6. Existence and Uniqueness of Solutions to the Major and Minor 
Stochastic Mean Field (SMF) System. In this section we establish existence 
and uniqueness for the solution of the joint major and minor (MM) agents' SMF 
system (|5.8p - (|5.10p and (I5.18[) - (|5.20[) . The analysis is based on providing sufficient 
conditions for a map that goes from the random measure of minor agents p^{u) back 
to itself, through the equations (I5.8[) - (|5.10[) and (|5.18p - (|5.20p . to be a contraction 
operator on the space of random probability measures (see the diagram below). 

(<fo(-,w,x), V , o(-,^,a;)) u% (-,uj,x ) 

t jooi 1 0a 

. |5.19[ , x ]5.18[ 

u°[-,u,x) (</>(•, u,x),ip{-,u}, x)) 

In this section we first introduce some preliminary material about the Wasserstein 
space of probability measures. Second, we analyze the SHJB and SMV equations of 
the major agent and minor agents in Sections 16.11 and 16.21 respectively. Third, the 
analysis of the joint major and minor agents' SMF system is carried out in Section [6. 31 
where the main result is given in Theorem 16.121 which provides sufficient conditions 
for a contraction operator map that goes from the random measure of minor agents 
/i(.)(w) back to itself. 

On the Banach space C([0, T\\ W 1 ) we define the metric pr(x, y) = sup 0<t<T \x(t)— 
y(t)\ 2 A 1, where A denotes minimum. It can be shown that C p := (C([0, T}; R"), px) 
forms a separable complete metric space (i.e., a Polish space). Let M(C P ) be the 
space of all Borel probability measures p on C([0, T]; R") such that / \x\ 2 dp(x) < oo. 
We also denote M(C P x C p ) as the space of probability measures on the product space 
C([0, T]; R") x C([0, T];R n ). As in [22] the process x is defined to be a generic random 
process with the sample space C([0, T]; R"), i.e., x(t,w) = w(t) for w e C([0,T];R"). 

Based on the metric pt, we introduce the Wasserstein metric on Ai(C p ): 

r f i 1 i 2 

D p T (p,v) = inf / p T {x{uj 1 ),x{uj2))di(u] 1 ,w 2 ) 
7en(/i,r/) Lj CpXCp J 

where n(/i, v) C Ai(C p x C p ) is the set of Borel probability measures 7 such that 
y(A x C([0,T];R")) = p(A) and 7 (C([0, T]; R") x A) = v{A) for any Borel set A G 
C([0,T];R"). The metric space M p := (M(C P ),D^) is a Polish space since C p = 
(C*([0,T];R"),p T ) is a Polish space. 

We also introduce the class of stochastic measures in the space M. p with 
a.s. Holder continuity of exponent /3, < /3 < 1 (see Definition 3 in [22] for the 
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non-stochastic case). 

Definition 6.1. A stochastic probability measure jtxt(w), < t < T, in the space 
M. p is in M.P if fi is a.s. uniformly Holder continuous with exponent < (3 < 1, 
i.e., there exists (3 G (0,1) and constant c such that for any bounded and Lipschitz 
continuos function ip on R™ , 

| / cj)(x)nt(u,dx) - / </)(x)h s (uj, dx)\ < c(w)\t - sf, a.s., 

JR™ JR™ 

for all < s < t < T , where c may depend upon the Lipschitz constant of <f> and the 
sample point lo G fl. 

As in |22j . we may take \it, < t < T, to be a Dirac measure at any constant 
x £ M" to show that the set M^ p is nonempty. We introduce the following assumption. 
(A8) For any p G M™ and /i, ,u G M?, the sets 

So(t,uj,x,p) := arg inf Hq° [t, u, x, u ,p), 

S(t, uj, x,p) := arg inf H u [t, lo, x, u,p], 

where Hq° and H u are respectively defined in (|5.5j) and ()5.15j) . are singletons and the 
resulting u and uo as functions of [t, lo, x,p] are a.s. continuous in t, Lipschitz contin- 
uous in (x,p), uniformly with respect to t and G -Mp- In addition, uo[t, lo, 0, 0] 
and lo, 0, 0] are in the space Ljr ([0, T]; R n ). 

The first part of (A8) may be satisfied under suitable convexity conditions with 
respect to u and u (see [22]). 

6.1. Analysis of the Major Agent's SMF System. Let < t < T, be 

a fixed stochastic measure in the set M." p with < (3 < 1 such that /J,o(dx) := dF(x) 
where F is defined in (A2). Then, the functionals of /j,^(lo) in (I5.1l) - (|5.2p become 
random functions which we write as 

(6-1) fo[t,(J,z ,u ] := fo[t,z ,u ,nt(u)], o-l[t,u,z ] := a [t, z , /x t (w)], 

Lg[t,w,z ,u ] := £o[£, z , u , /i t (w)]. 

We have the following result which broadly follows Proposition 4 in [2"2"] . 

PROPOSITION 6.2. Assume (A3) holds for U Q . Let Ht(w), < t < T, be a fixed 
stochastic measure in the set with < fi < 1. For / *, CTq and Lq defined in 16. 1\) 
it is the case that: 

(i) Under (A.4) for fo ando~o, the functions /q[^,o;, zq,uo] and <7q [t, to, zq] and their 
first order derivatives (w.r.t zq) are a.s. continuous and bounded on [0, T] x R ra x 
Uo and [0,T] xl". /q [t, lo, zq, uq] and CTq^w, Zq] are a.s. Lipschitz continuous 
in zq. In addition, /q [t, W, 0, 0] is in the space L?-p ([0,T];R") and aQ[t,Lo,0] is 
m the space L 2 jr t ([0,T];R nxm ). 
(ii) Under (A.5) for fo, the function fo [t, to, zq, uo] is a.s. Lipschitz continuous in 
uo G Uo, i.e., there exist a constant c > such that 

SU P \fo[t,u,z ,uo] - fo [t,LO,z ,u' }\ < c(lo)\u - u' \, (a.s.). 

*e[0,T],z GR™ 

(Hi) Under (A.6) for Lo, the function L^[t, lo, zq, Uq] and its first order derivative 
(w.r.t zo) is a.s. continuous and bounded on [0,T] x R" x Uo- L^[t, lo, zq, uo] 
is a.s. Lipschitz continuous in zq. In addition, L^[t, lo, 0, 0] is in the space 
L%([0,T];R + ). 
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(iv) Under (A8) for , the set of minimizers 

arg inf {(/£ [t, w, z , u ],p) + L* Q [t, u, z , u ] }, 

is a singleton for any p £ M n , and the resulting u$ as a function of [t, w, Zo,p] is 
a.s. continuous in t, a.s. Lipschitz continuous in (zo,p), uniformly with respect 
to t. In addition, Uo[t, w, 0, 0] is in the space Ly ([0, T];K n ). 
Proof: (i) We only show the results for /q, the analysis for ctq is similar. For 
lo £ fl, we take (t, z, u) and (s, z' , u') both from [0, T] x R" x Uq. We have 

\fu[t,w,z,u] - fg[s,oj,z,u'}\ = \f [t,z,u,fj,t{(j)] - f [s,z',u',fj, 3 (uj)]\ 

< \f [t,z,u,(it(u)] - /o[s,2',tt',/Ji(w)]| + |/o[s,z',w',^(w)] - fo[s,z',u',fJ, s (uj)]\ 

< |/o[M,w,// t (u/)] - /o[s,z,w, jti t (w)]| + |/ [s,^,u,Mi(w)] - /o[s,z',w',^ t (w)]| 

+ |/q[S)2'i«',^(w)] - /o[s,z',m',^ s (w)]. 

By (A4), /o[f, w, z, it] is continuous with respect to (t, 2, u) and therefore 

|/o[t, z,m, - / [s, z, u,/i t (o;)]| + \f [s,z,u,(j,t(uj)] - f [s,z',u'n t (w)]\ 0, 

as \t — s\ + \z — z'\ + \u — u'\ — > 0. Since /i(.)(u;) is in the set Mr p , < /3 < 1, and by 
(A4) there exists a constant k > independent of (s, z, u) such that 

\fo[s,z,u,y] - f [s,z,u,y']\ <k\y-y'\, 

we get \fo[s, z' ,u' , fi t {uj)] — f [s, z', u', /U s (w)] — > as |t — s| — > 0. This concludes the 
a.s. continuity of fg[t,uj, z Q ,u ] on [0,T] x M™ x C/ . 
Using the Leibniz rule we have 



D Zo fo[t,L),z , uo] = J D Zo f [t,ZQ,uo,x]n t (u))(dx), a.s., 

where the partial derivative exists due to the boundedness of the first order derivative 
(w.r.t zo) of f by (A4) . The a.s. continuity of -Dz /o on [0, T] x MP x U may be 
proved by a similar argument above for /q. Other results of the Proposition follow 
directly from (A4). 

(ii) This is a direct result of (A5). 

(Hi) The proofs are similar to the proofs for /q in part (i). 

(iv) This is a direct result of (A8) for So using the measure € 
< < 1. □ 

Employing the results of Section [31 we analyze the SHJB equation (I5.8P where 
the probability measure is in the set M?, < /3 < 1. 

Theorem 6.3. Assume (A3 ) -(A7) for Uq, fo, cto and Lq hold, and the probability 
measure fj,^(u>) is in the set MP p , < /3 < 1. XTien i/ie SHJB equation for the major 
agent < TO)) has a unique solution (cj) (t,x),ip (t,x)) in (L% t ([0, T]; R), L|- ( ([0, T]; M m )) . 
Proof. Proposition l6.2l indicates that the SOCP of the major agent (|5.ip - (|5.2p satisfies 
the Assumptions (H1)-(H3) of Section [3J with s[t,x] — 0. The result follows directly 
from Theorem 13. II □ 

Let G A-l^, < /3 < 1, be given. We assume that the unique solution 

((f>o,ipo)(t,x) to the SHJB equation (|5.8I) satisfies the regularity properties: (i) for 
each t, (4>o,Tjjo)(t,x) is a C 2 (R") map from M n into R x R m , (ii) for each x, (<£o,V>o) 
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and (D x (f>o, D^. x (f>o, D x iPq) are continuous -F^-adapted stochastic processes. Then, 
4>o(x,t) coincides with the value function (I5.3P [45] . and under (A8) for H^ we get 
the best response control process (|5.5[) : 

(6.2) v,Q(t,u>,x) = UQ(t,x\{/j, s (u})}o< s < T ) := arg inf Ho°[t,Qj,x,u ,D x <t>a(t,oj,x)], 

U &Jo 

where (t,x) G [0,T] x R". 

We introduce the following assumption (see (H6) in |22|). 

(A9) For any /i(.)(u;) G .M^, < /3 < 1, the best response control ug(i,u;,x) is 
a.s. continuous in {t, x) and a.s. Lipschitz continuous in x. 

We denote C , Li p ( a: )([0, T] x fl x M. n ;H) be the class of a.s. continuous functions 
from [0, T] x ft x M. n to H, which are a.s. Lipschitz continuous in x [22] . We introduce 
the following well-defined map: 

(6.3) T SHJB : Mf — > C Lip(x) ([0, T] x Q x R"; f/ ), 0< /? < 1, 
T SHJB (/i(.)H) = «o(t,w,aO = ttg(t > x|{/i.(a;)}o<.<r). 

We now analyze the major agent's SMV equation (|5.10[) with fx^(ui) G A4p where 
< P < 1, and u°{t,uj,x) G C Up(a .)([0, T] x Q x M n ;C/ ) be given in (j^2l . 

Theorem 6.4. Assume (A3)-(A7) for U , fo and a , and (A9) hold. Let 
/^(.)(cj) G M.P where < /3 < 1, and Uo(t, w, x) be given in \6. ty) . Then, there exists a 
unique solution Zq on [0,T] x fl to the major agent's SMV equation &5.10\) . 

Proof. Proposition 16.21 indicates that the major agent's SMV equation (|5. 10[) 
satisfies the Assumption (RC) in 54], page 49. The result follows directly from 
Theorem 6.16, Chapter 1 of [51]. page 49. □ 

Theorem 6.5. Assume (A3,)-(A7,) for U , fo and a , and (A9) hold. Let 
//(.) (uj) G M.P where < (3 < 1, and u^t, U), x) be given in 116. ty) . Then, the probability 
measure /j,®^(uj) obtained from the major agent's SMV equation H5.10\) is in the class 
MJ where < 7 < 1/2. 

Proof. We take < s < t < T. Since /i°(u;) = <5 2 °(t iLJ ), for any bounded and 
Lipschitz continuos function (f> on R™ with a Lipschitz constant K > 0, we have 

E\ f 4>(x)tf(u,dx)- [ </>(x)£(w,dx)\ =i?|0(z o o (t, W ))-0(z o o ( S , W ))| 

JR™ JR™ 

< K E\z°{t,uj) - z°(s,uj)\. 

On the other hand, Theorem 16.41 indicates that there exists a unique solution to the 
SMV equation ([5TT0]) such that 

Z o(^ U ) ~ Z o( S ^) = J Mr, ZQ,UQ,H T (uj)}dT + J o-o[T,ZQ,n T (uj)}dwo{T). 

Boundedness of fo and <To (see (A4)), the Cauchy-Schwarz inequality and the property 
of Ito integral yield 

E\z°{t,u) - z°(s,u)\ 2 < 2Cf\t - s\ 2 + 2C 2 \t - s\, 

where C\ and C2 are upper bounds for fo and o~o, respectively. Hence, 

E\ [ <f>{x)fi° t {cj,dx)- [ <f>{x)fj,° s {uj,dx)\<V2K(C 1 \t-s\+C2\t-s\ 1/2 ) 

JR« JK" 

< V2K{c 1 Vr + c 2 )\t- s\ 1/2 . 
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By Kolmogorov's Theorem (Theorem 18.19, Page 266, [IS]), for each < 7 < 1/2, 
T > 0, and almost every u € 57, there exists a constant c(uj, 7, if, T) such that 

|/ 4(x)i${w,dx)- [ 4>{x)/i a {w,dx)\<c(u,>y,K,T)\t-8\*', 

for all < s < t < T. Hence, f£Ju>) is in the class MJ where < 7 < 1/2. □ 
By Theorems 16 .41 and 16.51 we may now introduce the following well-defined map: 

(6.4) 

T SMV : x C Li p( x )([0, T]xflxl n ; U ) — > M], 0</3<l, 0< 7 < 1/2, 

t smv (// ( .) m^w,*)) =^.)M. 

6.2. Analysis of the Minor Agents' SMF System. Let /i(.)(w) e M?, < 
/? < 1, be the fixed stochastic measure assumed in Section l6.ll In this section we 
assume that S MJ, < 7 < 1/2, is the obtained stochastic measure of the 

major agent from the composite map: 

(6.5) T : Ml — > AP p , < p < 1, < 7 < 1/2, 

To (/*(.) H) ^To^^coM.To^^.jM)) =^.)M, 

where To SHJB and To SMV are given in (16.31) and (16.41) . respectively. 

Following arguments exactly parallel to those used in Section 16.11 we analyze the 
SHJB equation (|5.18[) where the probability measures (w) e M£, < /3 < 1 and 
«f.)(w)eMT ) 0<7<l/2. 

Theorem 6.6. Assume (A3)-(A.7) for U, f, a and L hold, and £t(.)(w) € 
M p p , < /3 < 1 and tfJu)) is in the set MJ, < 7 < 1/2. Then the SHJB 
equation for the generic minor agent \5. 1J$ has a unique solution {(pi{t,x),ipi{t,x)) 
m (L^ t ({0,nR),L%({0,nW™)). 

Proof. A similar argument to Proposition 16.21 for the generic minor agent (see 
Proposition IC.ll in [3 9) ) indicates that the SOCP of the generic minor agent (|5.1ip - 
(I5.12j) satisfies the Assumptions (H1)-(H3) of Section [3] with <j[t,x] = 0. The result 
follows directly from Theorem 13. II □ 

For the probability measure fi(.)(u)) £ < (3 < 1, and M(.)( w ) £ -MJ, 

< 7 < 1/2, we assume that the unique solution (<f>i,if)i)(t,x) to the SHJB equation 
(|5.14p satisfies the regularity properties: (i) for each t, (<pi,i/ji)(t,x) is a C 2 (W) map 
from E ra into K x R m , (ii) for each x, (4>i,ipi) and (D x 4>i, D^, x (j)i, D x ijji) are continuous 
-F^-adapted stochastic processes. Then, (f>i(x,t) coincides with the value function 
(|5.13[) [45] . and under (A8) for H u we get the best response control process (|5.15[) : 

(6.6) u°(t,uj,x) = u°(t, x\{fi° s {w), fi s (u)} < s < T ) 

:= arg inf H u [t,oj,x,u i ,D x 4> i (t,u,x)], 

where (t,x) £ [0,T] x R". 

We introduce the following assumption (see (A9) or (H6) in |22j). 

(A10) For any M(-)H G < (3 < 1, and ^(w) £ MJ, < 7 < 1/2, the 

best response control process u°(t,u>, x) is a.s. continuous in (t, x) and a.s. Lipschitz 
continuous in x. 
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We introduce the following well-defined map for the generic minor agent i: 
(6.7) 

t .shjb . M ff x M 7 c Up(x) ([0,T} x fi X R"; U), < /3 < 1, < 7 < 1/2, 
^^((j),^.,^)) = <(^,z) = u?(t )a: |{^(w) J ^( W )}o<.<r). 

For given probability measure (jVJuj) G -MJ, < 7 < 1/2, we analyze the generic 
minor agent's SMV equation (|5.16|) : 

(6.8) dz?(t,u,u') = f[t,z°,u° i (t,u,z?),$(u),» t (u>)]dt 

+ <r[t,zln° t (uj),(it(u)]dw i {t,uj'), z?(0) = z. ( (0), 

where u°(t,u},x) G C Lip(;r) ([0, T] x ft x E n ; 17) is given in (|Sj5]> . We call the pair 
(z°(-,u>,u>'), nr.\(w)) a consistent solution of the generic minor agent's SMV equation 
(16. 8p if (z°(-,u},u}'),[i(.)(u))) solves (|6.8[) and jU(.)(w) be the the law of the process 
z°(-, a/), i.e., = £(z°(-, uj, uj')) . We define A as the map which associates to 
G At£, < ft < 1/2, the law of the process z°(-,w,w') in (|6.8I) : 

(6.9) z°(t,u,u;') = z°(0) + [ ([ I f[s,z°^,y,z]d£{u){y)dM{z))ds 

c[s, z°, y, 2:]d/i"(cj)(y)d^(cj)(z) )dw l {s, u'), 



where we observe that the law A depends on the sample point u£fi. 

We now show that there exists a unique /i(.)(u;) G M.^, < /3 < 1, such that 
//(w) = A(/x(cj)). The proof of the following theorem, which is given in Appendix D 
in |39j . is based upon a fixed point argument with random parameters (see Theorem 
6 in [22] and Theorem 1.1 in [47] for the standard fixed point argument). 

Theorem 6.7. Assume (A3)- (A7) for U, f and a, and (AW) hold. Let fi9\(cj) 
be in the set MJ where < 7 < 1/2, and u°(t, lo, x) be given in H6.6}) . Then, there ex- 
ists a unique consistent solution pair (z°(-, w, u/), /x(.)(w)) to the generic minor agent's 
SMV equation 116.8]) where /j,t.\(cj) — C(z°(-, u), a/)) . □ 

Theorem 6.8. Assume (A3)-(A7) for U, f and a, and (A10) hold. Let 
^9x(o;) be in the set SAJ where < 7 < 1/2. For given u°{t,ui,x) in \6. 6}) . let 
(z°{-,uj, w'), H(.\(w)) be the consistent solution pair of the SMV equation \6.8\) . Then, 
the probability measure /l«(.)(cj) is in the class where < (3 < 1. 

Proof. We take < s < t < T. For any bounded and Lipschitz continuos function 
<f> on K™ with a Lipschitz constant K > 0, we have 



E\ I <j){x)iit(u,dx) - I <f)(x)fi s (u),dx)\ = E\E u ((j)(z°(t,uj,uj r )) - <f>(z°(s,u3,u')))\ 
< K E\E u (z°(t,LJ,u') - z?(a,u,u'))\. 



On the other hand, Theorem 16.71 indicates that there exists a unique solution to the 
SMV equation such that 

E u (zW,u,J)-%(a,u,<J))= f f[T,zf,ui,ti° T (u;),ti T (u)]dT, 



Mean Field Game Theory Involving Major and Minor Agents 21 

where we note that E u J Q a[r, z°, fJ, T (u))]d,Wi(T, u)') — for < t < T. Bound- 

edness of / (see (A4)) yields 

ElE^ztfautu/) - z?(a,u,u/)) \ < Ci\t-a\, 

where C\ is the upper bound for /. 

By Kolmogorov's Theorem (Theorem 18.19, Page 266, [53], Page 266), for each 
< 7 < 1, T > 0, and almost every cj S fi, there exists a constant c(ui, 7, K, T) such 
that 

I / (j>(x)fi t (uj,dx) - / (/>(x)Hs(uj,dx)\ < c(w,7, K, T)\t - s| 7 , 

for all < s < t < T. Hence, (J,(.)(u) is in the class M% where < fi < 1. □ 
By Theorems 16 . 71 and 16 .81 we may now introduce the following well-defined map: 

(6.10) 

T, SMV : M; x C Lip(x) ([0,T] x!lx R»; (7 ) — ► M$, < /3 < 1, < 7 < 1/2, 
T i SMV (^. ) H,«?(t, W ,x))=/i ( . ) (a;). 

6.3. Analysis of the Joint Major and Minor Agents' SMF System. 

Based on the analysis of Sections 16.11 and 16.21 we obtain the following well-defined 
map: 

(6.11) T : Mjf — > Aff , < [3 < 1, 
T( A1( . ) H)=T,; SMV (T ( M( . ) H),T l SHJB ( A1( . ) H),T ( A1( . ) H))), 

which is the composition of the maps To, Tf HJB and T^ MV introduced in (|6.5I) . (I6.7|) 
and (|6.10p . respectively. Subsequently, the problem of existence and uniqueness of 
solution to the MM SMV system (pT8>([5T0j) and (|5T8| - ([5?20| is translated into a 
fixed point problem with random parameters for the map T on the Polish space 
< p < 1. 

We introduce the following assumption without which one needs to work with the 
"expectation" of the Wasserstein metric D?~. of stochastic measure. 

(All) We assume that the diffusion coefficient of the major agent Co in (jUTJ does 
not depend on its own state Zq and the states of the minor agents zf , 1 < i < N. 

The proof of the following lemma is given in Appendix E in [39] . 

Lemma 6.9. (i) Assume (A3)-(A7) for U , fo and a , and (All) hold. Let 
/i(.)(o;) be in the set Mr where < /3 < 1. Then, for given Uq,u' q G Cxj p ( x )([0, T] x 
f2 x R™;C7o) there exists a constant cq such that 

(6.12) (d^ < c sup \u a (t, uj, x) — u' (t, w, x) | 2 , a.s., 

where /jP(oj), ^°(w) € -MJ, < 7 < 1/2, are induced by the map Tq MV in {6.$ using 
the two control processes uq and u' , respectively. 

(ii) Assume (A$)-(A7) for Uq, fa and ao, and (All) hold. Let Uq be in the space 
Cli P (x)([0,T] xflx K"; U ). Then, for given fi(u>), v(w) € M.%, < (3 < 1, there exists 
a constant C\ such that 

(6.13) (D^(n°(uj),v (uj))y < d(z>£(p(w),i/(w))) a , a.s., 
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where /j,°(ui), v®(uj) G M^ p , < 7 < 1/2, are induced by the map Tq MV in |6'.4| i using 
the stochastic measures h{uj) and v{uj), respectively. 

(Hi) Assume (A3)-(A7 ) for U, f and a hold. Let be in the set A4J where 

< 7 < 1/2. Then, for given u, u' g Cu p ^ x -)([0, T] x fl x R"; U) there exists a constant 
C2 such that 

(6.14) \D P T (//(w),i>(u>)) ) < C2 sup |u(t, a;) — u'(t, oj, a;) | 2 , a.s., 

^ ' (t,x)e[0,T]xR" 

where [i{uj),v{uj) € -M^, < /? < 1, are induced by the map Tf MV in \6.1Q\) using 
the two control processes u and u' , respectively. 

(iv) Assume (A3)-(A7 ) for U, f and a hold. Let u° be in the space C^ ip ^ ([0, T] x 
x R"; {/). Then, for given (j, (uj), v°(uj) €E A^, < 7 < 1/2, £/iere exists a constant 
C3 smc/i i/iai 

(6.15) (^( M (u;),^))) 2 <C3(^(M (a;)^ (a;))) 2 , 

where h(uj),v(uj) G Af^, < /3 < 1, are induced by the map Tf MV in i6.10\) using 
the stochastic measures /j,°(uj) and v°(uj), respectively. □ 
We define the Gateaux derivative of the function F(t,x,fi) with respect to the 
measure /i(y) as [25] 

d, (y) F{t,x,») = lim + 

where 5 is the Dirac delta function. We introduce the following assumptions: 

(A12) (i) In ([5.1[) - (l5.2j) the Gateaux derivative of fo, cr and L with respect 
to /1 exist, are C°°(R™) and a.s. uniformly bounded, (ii) In (|5 . 1 1 [) - (|5 . 1 2[) the partial 
derivatives of /, a and L with respect to /j? and /i exist, are C°° (R") and a.s. uniformly 
bounded. 

The proof of the following lemma is based on the sensitivity analysis of the SHJB 
equations (|5.8|) and (|5.18[) to the stochastic measures /i(.)(w) and ^^(lu) developed in 
Appendix F in [3j5] (see also Section 6 in [25]). 

Lemma 6.10. (i) Assume (A3)-(A7) for U , fo, L , and (AYl)-(i) hold. 
Let (</>o(t, x), ipo(t, x)) be the unique solution pair to A5.8\) which is C°°(R ra ) and is a.s. 
uniformly bounded. In addition, we assume (A8) holds for So and the resulting uq 
is also a.s. Lipschitz continuous in fi. Then, for /i(.)(w) and v^.){uj) in the set M.^, 
< ft < 1, there exists a constant C4 such that 

(6.16) sup \uo{t, uj, x) — u' (t, uj, x) | 2 < cA Z?£(/i(w), v{uj)) ) , a.s., 

where u ,u' € Cu p t x \(\p,T\ x ft x M™;[/ ) are induced by the map Tq ,b in Iff.ffl 
using two stochastic measures fir,\(u)) and respectively. 

(ii) Assume (A3)- (A7) for U , f, a, L, and (AYl)-(ii) hold. Let (<fi(t, x), ip(t, x)) be 
the unique solution pair to \5. 18\) which is C°°(R") and is a.s. uniformly bounded. 
In addition, we assume (A8) holds for S and the resulting u is also a.s. Lipschitz 
continuous in /1. Then, for jj,9-<(uj) € MJ, < 7 < 1/2, and /j,^(uj) and v^{uj) in 
the set M? p , < [3 < 1, there exists a constant C5 such that 

(6.17) sup \u(t,uj,x) — u'(t,uj,x)\ 2 < C5 ( £>j,(/x(o;), v(uif) J , a.s., 

(t,x)e[0,T]xR» ^ ' 
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where u, v! € Clip(x)([0) T] x ft x R"; J7) are induced by the map Tf HJB in J6'. 7| ) using 
two stochastic measures fj,/.\(ui) and vi.\(w), respectively. 

(Hi) Assume (A3)- (A7) for U , f, a, L, and (AYl)-(ii) hold. Let (<fi(t, x), ip(t, x)) be 
the unique solution pair to i f 5. 18\) which is C°°(M. n ) and is a.s. uniformly bounded. 
In addition, we assume (A.8) holds for S and the resulting u is also a.s. Lipschitz 
continuous in fi° . Then, for € Mp, < /3 < 1, and ^^(uj) and v?Juj) in the 

set Ml, < 7 < 1/2, there exists a constant Cq such that 

(6.18) sup \u(t, uj, x) — u'(t, u>, x)\ 2 < cq ( Z?^(/i°(cj), v°(u>)^ ) , a.s., 

(t,i)e[o,r]xi» ^ ' 

where u, u 1 £ Ci, ip ^{[Q, T] x ft x R"; U) are induced by the map Tf HJB in J6'. 7| ) using 
the two stochastic measures /Lt^(w) and v^{ui), respectively. 

Proof, (i) Assumption (A8) for So together with the fact that the resulting uo in 
(A8) is also a.s. Lipschitz continuous in /i yields 

\u (t,uj,x) - u' (t,uj,x)\ < kiD^{/j,(oj),i/(oj)) 

(6.19) + k 2 \D x ^(t, u, x) - D x r (t, u,x)\, 

with positive constants k\,k 2 , where we indicate the dependence of (f>o on measures /x 
and v by 0q and </>q, respectively. 

We consider the Gateaux derivative of <po with respect to the measure [i. The 
assumptions of the theorem imply that the conditions for Proposition lF.ll in [39 hold. 
Therefore, Proposition lF.ll in [3T?] concludes that the Gateaux derivative of D x (f>o with 
respect to measure \x is a.s. uniformly bounded. This together with the mean value 
theorem yields 

(6.20) \D x( %(t,u,x)-D x <%(t,u,x)\ < fc 3 A P (^M,^H), 
with positive constant k 3 . (|6.19p and (|6.20l) give 

\u (t,uj,x) - u' (t,u),x)\ < kD p t (n{u),v{uj)), 

with k := k\ + k 2 k 3 , which yields the result. □ 
Remark 6.11. In the standard mean field game model of '221 a similar condition 
to i6.16]) - £6.18\) is taken as an assumption (see the feedback regularity condition (37) 
in J 2 2}/). Following the argument in Section 7.1 of [221, one can show that the inequal- 
ities /i6.16\) - ![6.18\) hold in the linear-quadratic-Gaussian (LQG) model with Lipschitz 
continuous nonlinear couplings. 

We recall the map T given in (|6.11|) which is the composition of the maps To, 
Tf HJB and Tf MV introduced in (H3]), $£7$ and ([6~TU1) . respectively. 

Theorem 6.12. (Main Result J Let the assumptions of both Lemma \6. 9\ and 
Lemma\6jEhold. If the constants {c t : < i < 6} for Kl2\) - ![6J5\) and lffJff)) - fOg)) 
satisfy the gain condition 

max{c 2 c 5 , c 2 c 6 c , c 2 c 6 ci, c 3 c 1: C3C0C4} < 1, 

then there exists a unique solution for the map T , and hence a unique solution to the 
MM SMF system fO )) - fOfl|) and fO^ - jTOfl)) . 

Proof. The result follows from the Banach fixed point theorem for the map T 
given in (|6.11|) on the Polish space M p p , < /3 < 1. We note that the gain condition 
ensures that T is a contraction. □ 
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7. e-Nash Equilibrium Property of the SMF Control Laws. We let 

((/)o(i,w,x)>o(i,w,x),Mg(t,a;,x),Zo(t,a;),(?!)(t,a;,a;),V'(i,w,a;),w (t,w,a;),z (t,a;)), 

be the unique solution of the MM SMF system (|5^1) - (|5?TU1) and fl!Ug)) - (lO0)) such that 
the best response SMF control processes uj, x) and u°(t, oj, x) are a.s. continuous 
in (t, x) and a.s. Lipschitz continuous in x. 

We now apply the SMF control processes ttg(t, and u°(t,ui,x) into a finite 
N + 1 major and minor population (I2.1[) - (|2.2j) . This yields the following closed loop 
individual dynamics: 

1 N 

(7.1) dz° > N (t) = jjYMtX' N ®,<M' N m% WW 

3 = 1 

1 N 

+ - £ a [t, z° > N (t),z°> N (t)]dw (t), z° ' N (0) = z (0), < t < T, 

3=1 

1 N 

(7.2) dz? N (t) = -Y J f[t^T N {t),u {t,zT N {t)lz^ N {tlz°f N {t)}dt 

3 = 1 

1 N 

+ ®X' N (t),*j '"(Ww^t), z°> N (0) = Zi (Q), 1 < * < N, 

3 = 1 

We set the admissible control set of agent Aj, < j < N, as 

Uj = {uj(-,u) := Uj(-,u3,z (-,uj), ■ ■ ■ ,z N (-,u)) € C Up (z ,- ,z N ) '■ Uj(t,u) is a 

-measurable process adapted to sigma-field a{zi(r, uj) : < i < N, < r < 

such that E J \uj(t, Lo)\ 2 dt < oo|. 

We note that Uj, < j < N, are the full information admissible control which are 
not restricted to be decentralized. 

Definition 7. 1 . Given e > 0, the admissible control laws (uq, • • • , u° N ) for N + 1 
agents generates an e-Nash equilibrium, with respect to the costs , < j < N, if 
Jf («°; u^) - e < inf MjG ^. ./ ( v ( ,/,: < Jf (u°; u^), /or any < j < N. □ 

We now show that the SMF control processes for a finite N + 1 major and minor 
population system (|7.ip - (|7.2|) is an e-Nash equilibrium with respect to the cost func- 
tions (|2.3[) - (|2.4D in the case that minor agents are coupled to the major agent only 
through their cost functions (see the MM LQG model in [37]). 

(A13) Assume the functions / and a in (|2.2j) (and hence in (|7.2jl ) do not contain 
the state of major agent Zq . 

Note that in the case of assumption (A13) the major agent Aq has a significant 
influence on the minor agents through their cost functions (|2.4p . We note that an 
analysis based on the anticipative variational calculations used in the MM LQG case 
[38] is required for establishing the e-Nash equilibrium property of the SMF in the 
general case. 

Theorem 7.2. Assume (Alj-fAGj and (A13) hold, and there exists a unique 
solution to the MM SMF system fO)) - ifO^I) and ifO^)) -( f07J|) such that the SMF 
best response control processes UQ(t,uj,x) and u°(t,uj,x) are a.s. continuous in (t,x) 
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and a.s. Lipschitz continuous in x. Then (ug, u°, ■ ■ ■ , u° N ) where u° = u° , 1 < i < N , 
generates an 0{tN + 1/ VN)-Nash equilibrium with respect to the cost functions 12. 
such that limjv->oo ejv = 0. 
Proof. Under (A13) we have the the following closed loop individual dynamics 
under the SMF best response control processes: 



dz °o' N ® = iE/o[*.*o ,JV (*).«o(* 5 z° > N (t)),z°> N (t)]dt 

3=1 

1 N 

+ ^J2^[t,4' N (t),z^ N (t)]dw (t), z° N (0) = z (0), < t < T, 
1 N 

dz ° ' (*) = ^E/tMf^MXM'^)),*; ' N it)]dt 

3 = 1 

1 - 

+ -^,[Mf(i),^>W, v>) = *(o), l < i < jv. 



We also introduce the associated Mckean- Vlasov (MV) SDE system 

(7.3) dz°{t) = f Q [t, z°(t), u°(t, z°)^ t ]dt + a [t, z°(t), fi t ]dw (t), 
dz°(t) = f[t, z°(t), u°(t, z°), n t }dt + a[t, z°, (i t ]dwi(t), 

with the initial condition Zj(Q) = Zj(0), < j < N. In the above MV equation /x t , < 
t < T, is the conditional law of z?(t), l<i<N, given J 7 ™ (i.e., fi t := £(z°(t)\ , 
1 < i < N). Theorem |4~T1 implies that 

(7.4) sup sup E\z°' N {t)-z°At)\ = 0{l/y/N), 

0<j<N 0<t<T 

where the right hand side may depend upon the terminal time T. 

Let z(0) = J M „ xdF(x) be the mean value of the minor agents' initial states (see 
(A2)). We denote 

{e N f = j x T xdF N (x) - 2z T (0) [ xdF N (x) + z T (0)z(0) 
it" J& N 

It is evident from (A2) that limjv->oc £n = 0. To prove the e-Nash equilibrium 
property we consider two cases as follows. 

Case I (strategy change for the major agent Aq). While the minor agents are 
using the SMF best response control law u°(t, uj, x), a strategy change from ito(i, w, x) 
to the J r t 1 " -adapted process wo(i,w,x, z°_1q {t,u))} € Uq for the major agent yields 



1 N 

3 = 1 

1 N 

+ ^J2 a ^ z o( t )> z ? N ( t )] dw o(t), z»(0) = z (0), 0<i<T, 



where z"'^ = (z° :N , ■ ■ ■ ,z^ N ). Since minor agents are coupled to the major agent 
only through their cost functions (see (A13)) the strategy change of the major agent 
does not affect the the minor agents' states z°' and z°, 1 < i < N, above. 
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Let z^(-) be the solution of the SDE: 

JV 

N 



3=1 
1 N 

+ ^J2 a ^ 2 o(t)^jit)}dw (t), 2^(0) = zo(0), 



TV 

where z° = (z°, • • • , z^) is given by the MV SDE system above. Theorem 14.11 and 
the Gronwall's lemma imply that 

(7.5) sup E\z?(t) - z?{t)\ = 0(1/VN). 

0<t<T 

We also introduce the SDE 

dz (t) = f [t, z (t) ,u (t, z a (t) , z°_ o (t) ),/j lt \dt + a [t,z (t) , n t ]dw (t) , 

with initial condition Zq(0) = Zq(0), where fj,/A is the minor agents' measure given by 
the MV SDE system above. Again, by Theorem 14.11 and the Gronwall's lemma It can 
be shown that 

(7.6) sup E\z?(t)-zo(t)\ = 0(l/VN). 

0<t<T 

(A3), (A6), (H3^([L!1) and Theorem O yield 

(7.7) J^(u ;u°_ )=E / [(l/N)J2Lo[t,z^t),u (t,z^z°:^, z ;' N (t)})dt 
Jo j=1 

E / ((l/N)J2Lo[^4(t)^o(t,z^z°_ ),z°(t)]yt-0(e N + l/VN) 
j= i 

N 

\l/N) J2 Lo [t, %o(t), uo(*, z°_ ),z°(t)])dt - 0(e N + 1/Vn) 



> 

3=1 

> E 



3 = 1 



~ 2^ [t,^(t),«o(t,^,«l ).*j (*)])* -0(ew + VV^ 



> E 
133 



> 



£ / L [Mo(i), uo(Mo, 2° ),^]^-O(e A r + l/V / iV), 



where the appearance of the e^v term in the first inequality of (|7.7[) is due to the 
fact that here the sequence of minor agents' initials {z°(0) : 1 < j < N} in the SMV 
system (17.31) is generated by independent randomized observations on the distribution 
F given in (A2). 

Furthermore, by the construction of the major agent's SMF system (|5 . 8[) - (|5 . 101) 
(see the major agent's SOCP (|5.1|) - ()5.2|) ) we have 

(7.8) e[ L Q [t,z {i),u {t,z Q ,z°_ Q ),ixt]dt>E{ L [t, z°{t),u° Q {t, zg), fi t ]dt. 
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But, Theorem |4~TI and (I7.4j) imply 
(7.9) 

E ( L [t,zZ(t),u° (t,zZ),m]dt 
Jo 

ED 



> E f(l/jV)5^L [t,^(«),uo(« ) zg),«;(t)]Jdt-0(e J v + l/V]V) 

,/0 3=1 

> £ / ((l/iV)^L [i,z ^(0, M o(i,^ JV ) ) 4'' V W])*-0(e w + l/^) 



3=1 
= J Ar K; U °_ )~O( ejv + l/ViV). 

It follows from ^IJ\-§TM) that J^(itg;ui ) - 0{e N + 1/VN) < inf„ oeWo Jrf (u ; w°_ ). 

Case II (strategy change for the minor agents): Without loss of generality, we 
assume that the first minor agent changes its MF best response control strategy 
u°(t, to, x) to u\ (t, uj, x, z^i(t, a;)) eWj.. This leads to 

N N 



AT W 

d ^ W = ^ E *-i)> # ^ + ]v= E <^> *i,z?]dwi®, 

3 = 1 3=1 

= -E/^^^ o (i,^),#]*+^E CT ^^^f]^ 2 (i), 



3=1 3=1 



JV JV 

dz%® = - E /[*, 4> «°(*, O, *f + E *#, *f 

3=1 3=1 



By the same argument as in proving Theorem 14.11 (see Appendix B in 39 ) it can be 
shown that 

sup sup E\z°' N (t)-z?(t)\=OQ./VN), 

3=0,2, ••• ,N 0<t<T 

sup sup E\z°At) - zf (t)| = 0(1/VN), 

3=0,2,." ,N 0<t<T 

Let if (•) be the solution of the SDE: 
1 N 

3 = 1 

1 N 

+ ^E CT Mf (*W(*)]^i(*), ^(°) = z i(°)' o < < < T, 

3=1 
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where z°_ x = (z°, • • • , z^) is given by the MV SDE system above. Theorem 14.11 and 
the Gronwall's lemma implies that 

(7.10) sup E\z?(t) - (t)| = 0(1/ VW). 

0<t<T 

We also introduce the SDE 

dzi(t) = f[t, zi(t), ui(t, zi(t), z°_ x (t)), nt]dt + a[t, h(t) ,fx t ]dwi(t) , 

with initial condition Zi(0) = zi(0), where fir.) is the minor agents' measure given by 
the MV SDE system above. Again, by Theorem 14.11 and the Gronwall's lemma It can 
be shown that 

(7.11) sup E\z? (t) - Zl (t)\ = 0(1/ Vn). 

0<t<T 

Using (17.41) and (|7.10[) - (|7.11[) . and by the same argument as in (I7.7[) - (|7.9[) we can 
show that JfK; u°_ x ) - 0(e N + 1/VN) < mi ueUl Ji(uv, !*_i). □ 

8. Conclusion. This paper studies a stochastic mean field (SMF) system for a 
class of dynamic games involving nonlinear stochastic dynamical systems with ma- 
jor and minor (MM) agents. The SMF system consists of coupled (i) backward in 
time stochastic Hamilton- Jacobi-Bellman (SHJB) equations, and (ii) forward in time 
stochastic McKean-Vlasov (SMV) or stochastic Fokker-Planck-Kolmogorov (SFPK) 
equations. Existence and uniqueness of the solution to the MM SMF system is es- 
tablished by a fixed point argument in the Wasserstein space of random probability 
measures. In the case that minor agents are coupled to the major agent only through 
their cost functions, the e^y-Nash equilibrium property of the SMF best response con- 
trol possess is shown for a finite N population system where e^v = 0(l/s/N). As a 
particular but important case, the results of Nguyen and Huang [37] for MM-SMF 
linear-quadratic-Gaussian (LQG) systems with homogeneous population are retrieved, 
and, in addition, the results of this paper are illustrated with a major and minor agent 
version of a game model of the synchronization of coupled nonlinear oscillators [50] 
(see Appendices G and H in [39]). 
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Appendiceal 

Appendix A: Extended Ito-Kunita Formula. We recall an extended version 
of the Ito-Kunita formula [37] for the composition of stochastic processes (see Theorem 
2.3 in @5]). 

Theorem A.l. Let <p(t,x) be a stochastic process a.s. continuous in (t,x) such 
that (i) for each t, <f>(t, •) is a C 2 (R") map a.s., (ii) for each x, </>(•, x) is a continuous 
semi-martingale represented by 

ra 

d<f>(t,x) = -T(t,x)dt + ^ / ip k (t,x)dW k (t), (t,x)e [0,T]xM™, 

fc=l 

where T(t,x) and ipk(t,x), 1 < k < ra, are -adapted stochastic processes which 
are continuous in (t,x) a.s., such that for each t, T(t, •) is a C 1 (R") map a.s., and 
ipk(t, ), 1 < k < m, are C 2 (M. n ) maps (a.s.). 

Let x(-) = • • • , x n (-)) be a continuous semi-martingale of the form 

ra m 

dx\t) = fi(t)dt + J2 a ^{t)dW k (t) + J2^{t)dB k {t), l<i<n, 

k=l k=l 

where fi, Oi = (an,-- - ,cr im ) and ft = (fti,--- ,ft m ), 1 < i < n, are -adapted 
stochastic processes such that (i) fi is an integrable process a.s., and (ii) er,; and ft are 
square integrable processes (a.s.). 

Then the composition map <f>[-,x{-)) is also a continuous semi-martingale which 
has the form 

ra n 

(A.l) #(t, x{t)) = -r(t, x{t))dt + V'fc (t, x(tj) dW k (t) + d*Mt> &(t)) /< (*)* 

fe=l 1=1 

n ra n ra 

i=l k=l i=l fe=l 

n ra ^ n tn 

z__l fc=i i,j=l k=l 

^ n m 

i,j=l fc— 1 

□ 

Appendix B: Proof of Theorem 14.11 (Mean Field Convergence Theo- 
rem). We will show 

sup sup E\zf{t) - Zj{t)\ 2 = 0(1/N), 

0<j<N 0<t<T 



1 This document supplies appendices of the paper "e-Nash Mean Field Game Theory for Nonlinear 
Stochastic Dynamical Systems with Major and Minor Agents" by Mojtaba Nourian and Peter E. 
Caines submitted to SIAM J. Control Optim. in Aug. 2012. 
Available at https : //www. dropbox . com/s/cy5xh956khd31zq/ Appendices .pdf 
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which implies the result of the theorem by the Cauchy-Schwarz inequality. First by 
the inequality (x + y) 2 < 2x 2 + 2y 2 , we have 



E\z»(t)-zo(tW 



< 2E 



f* ( 1 N \ 
/ {jj^2M s ^o^o(s,z^),z^] - fo[s,z ,(p (s,zo),fi s ])ds 



+ 2E 



1 N 



3=1 



a Q [s,ZQ,zf] - a [s,z ^ s ]jdw (i 



By the Cauchy-Schwarz inequality and the properties of ltd integrals we then obtain 



(B.l) E\z£(t)-z (t)\ 



< 



( f* I 1 N 

2tE \ J \^Ylj ^ s ^o ^o{s,z^),zf] - f [s,z ,<po(s,z ),i 



ds 



a [s,z ,fj, s 



ds 



Clearly, 



1 N 



3 = 1 



= ^°[ s ' 5 o i Po(s, *o ), % ] - — ^ f [s, z , <fio(s, z ),z j ]) 

3 = 1 3=1 
1 N 1 N 

+ (77 XI ^°[ s ' ^°( s ' ^),zf] - — ^ fo[s, zo, <po(s, Z ), Zj]} 

3=1 3=1 
1 N 

+ (tv^ /o[Mo,Vo(s, 20), %] - /o[Mo,V>o(s,2o),A« s ]), 



and 



i=i i=i 3=1 

1 N 1 N 1 N 

+ 'Y J oo[s,zo,zf] - — ^2ao[s,ZQ,Zjfj + (— ^2a [s,z ,Zj] - cr [s, z , fi s ]j ■ 



3=1 



3=1 



3=1 



Applying the inequality (x + y + z) 2 < 3(x +y +z ), and the Lipschitz continuity 
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conditions of / and ip to (|B.2|) we obtain 



rt , 1 AT 



(s,Zo),zf] - f [s,z Q ,(p Q (s,z Q ),fi s 



(B.3) £?(/" |I^/ [ s ,^^o( 6 

<3C/ £;|z^(s)-z (s)| 2 ds + 3C / #| — Vzf (s)-%(s)| 2 ds 

7o I ^ . =1 



(is, 



where C > is a constant independent of AT. Due to the centring of g s [s,z 0l x] := 
fo[s, zq, <po(s, zq),x] — fo[s, zq, <po(s, zq), //„] with respect to x and the independence of 
Zj and Zj> when j ^ j', there are no cross terms in the expansion of the last term in 
(|B.3|), i.e., E(g s [s, z Q , Zj]g s [s, z Q , z r }) = EE^o (g s [s ,z , Zj]g s [s, z , %']) = for j ^ j' 
(see [17], Page 175). This property together with (|B.3[) . the boundedness of /o and 



the inequality (Xh=i x if < ^J2tLi x \ yields 

,, AT 



(B.4) £?f / I — ^/oIsj^^oCs,^),^] - /o[s,2o,<po(s,2o),M»] 
j=i 

3r /•* N 

3=1 



<3C / E z^( S )~z (s) 



N ' 



where k\(t) > is an increasing function independent of N . Similarly, for the second 
term on the right hand side of (jB.lj) we have 



ds 

t N 



(B.5) E^J ^Y^a [s,ZQ,zf}-ao[s,z ,H ; 

< 3CJ* E\z»(s) - z (s)\ 2 d s + ^ ^ E 00 ~ ^( s )! 2ds + 



N ' 



The inequalities (jB.ll) . (|B.4|) and (|B.5|) imply that 
(B.6) sup E\z£(t)-z (t)\ 2 <6C(T+l) [ E\z$ (s) - z {s)\ 2 ds 

0<t<T Jo 



6C(T + 1) 
N 



[ Y^EVzf^-z^ds- 
Jo j=l 



2(T+ l)fci(T) 
N 



Second, by taking a similar approach for the i minor agent (1 < i < ./V) we get 



(B.7) sup £|if (t)-Zi(Or < 8C(T+1) / £|if (s) - ^(s)rda + 



0<t<T 



fc(T) 
N 



8C(T + E\z»(s) ~ z (s)\ 2 ds + 1 jT £ (s) - 
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where k(T) > is independent of N. 

The inequalities (|B.6I) and (IB.7|) yield 



(B.8) g N (T):= sup E\z»(t)-z (t)\ 2 + ^J2 sup E\zf (t) - z 3 (t)\ 2 

0<t<T JV , 0<t<T 
3=1 

<22C(T+1) / (i?|z A, ( S )-z ( S )| 2 + -^S|zf(.s)-z,( S )| 2 )d S 

J ° 3=1 

+ Mn±m < 22C(T + 1) r ff(s)ds + w+ffi . 

N J N 

It follows from Gronwall's Lemma that 

(B.9) g N (T) < fc ° (r) + fc(r) (exp (22C(T + 1)T)) = 0(l/N), 

where the right hand side may only depend upon the terminal time T . This yields 

sup E\z£ (t) - z (t)\ 2 =0(1/N). 

0<t<T 

The inequalities (|B.7I) and ()B.9|) combined with Gronwall's Lemma imply that 
sup sup E\zf(t) ~ I;(t)| 2 = 0{l/N). 

l<i<N 0<t<T 

This completes the proof. □ 

Appendix C. We may write the functionals of fi^Ju) and in (|5.11|) - 

()5.12j) as random functions: 

(C.l) f*\t,w,Zi,Ui] := f[t,Zi,Ui,Ht(uj),fit(uj)}, a*[t,u,Zi] := a\t,Zi,$(u)),n t (w)], 
L*[t,(J,Zi,Ui] := L[t,Zi,Ui,fi°(uj),fit(u)}- 

We have the following proposition where its proof closely resembles that of Propo- 
sition E21 (see Proposition 4 in |22|). 

Proposition C.l. Assume (A3) holds for U. Let (j, t (u)) , < t < T , be a fixed 
stochastic measure in the set Ai^ with < (3 < 1, and ^ j(oj) = To(/i(.)(o;)) g -MJ, 
< 7 < 1/2, be the obtained probability measure of the major agent in Section \6.1\ 
For f* , a* and L* defined in \C.1\) we have: 
(i) Under (AA) for f and a, the functions f*[t, ui, Zi, Ui] and a*[t,UJ,Zi] and their 
first order derivatives (w.r.t Zi ) are a.s. continuous and bounded on [0, T] x K™ x 
U and [0,T] xM". f*[t,uj, Zi,Ui] and o~*[t,Lu, Zi] are a.s. Lipschitz continuous in 
Z{. In addition, f*[t,u, 0,0] is in the space L?-p ([0,T];R n ) and <j*[t,w,0] is in 
the space L 2 jr t {[0,T];R nxm ). 
(ii) Under (A5) for f, the function f*[t, U), Zi, Ui] is a.s. Lipschitz continuous in 
Ui G U , i.e., there exist a constant c > such that 

sup \f*[t,u,z l ,u i ]- f*[t,u,z l ,u' i \\< c(uj)\ui- u'i\, (a.s.). 

te[0,T],Zi£lSL n 

(Hi) Under (A6 ) for L, the function L* [t, ui, 2j, u%] and its first order derivative (w.r.t 
Zi) is a.s. continuous and bounded on [0,T] x R™ x U. It is a.s. Lipschitz 
continuous in z.^. In addition, L*[t,u, 0,0] € hy ([0, T]; K+). 
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(iv) Under (A8) for H u , the set of minimizers 

arg inf {(/*[*, uj, z u Ui],p) + L* [t, u, z h Ui] }, 

is a singleton for any p G W 1 , and the resulting m as a function of [t, u>, Zi,p] is 
a.s. continuous in t, a.s. Lipschitz continuous in (Zi,p), uniformly with respect 
to t. In addition, Ui[t,co, 0,0] is in the space L 2 -p ([0, T]; R n ). 

□ 

Appendix D: Proof of Theorems 16.71 Let ui e il be fixed. For given prob- 
ability measure fx^(ui) € M.^ p , < (3 < 1, we can show that the law of the process 
z°(-,w,w') given in (|6J|) . A(z° (■ , oj , oj')) , belongs to M$, < /3 < 1 (see Theorem 



We take v (-)( u ) e -Mp> < /3 < 1. Let z°(-,u,u)') be defined by 

and similarly a:°(-,a;,u/) be defined by (|6.9p after replacing /i(.)(u;) by ^(.)(cj). We 
have 

(D.l) E K o sup |z°( S ,c)-x°( S , W )| 2 

0<s<t 



< 2t 



f[s, z°, ui, y, z]dfi° s (uj)(y)d[j, s (uj)(z) 

f[s, x°, ui, y, z]dy° s (u)(y)dv s (uj)(z) 

" xR™ 

z°, y, ^]^°(w)(y)^ s (w)(z) 



a[s, x°, y, z]dfi° s (uj)(y)dv s (u)(z) 



ds. 



But, 



fi s i z °i u i, V, z ] d M°(w)(y)(Ww)(z) - / f[s, <, ui, y, z]dfi° s (uj)(y)dv s (uj)(z) 



< 2C 



(\z°(s)-x1(s)\ 2 + [ |z s ( Wl )-z s (c 2 )| 2 d 7 ( Wl ,c 2 )), 



where C is obtained from the boundedness and Lipschitz continuity of both / and u° , 
and 7 € M(C P x C p ) is any coupling of fi and v where ^{A x C([0, T]; R™)) = /j,(A) 
and 7(C([0,T];R") x A) = v(A) for any Borel set A e C([0, T];R"). Taking the 
infimum over all such 7 couplings and then using the definition of metrics and 

(■) 

(D.2) 



D p , > yields 



/[si 2°, w°, 2/, 2]d^°(w)(?/)d^ s (w)(z) - / /[s, x°, u°, y, z]d£{uj){y)di> s {uj)(z) 
<2c(p s (z°(s),x1(s)) + (D^,v)) 2 ). 
Similarly we have 

(D.3) J o-[s,z°,y,z\d(jP 8 (u)(y)dfJ, s (u)(z) - J cr[s,x1,y,z}dfi s (uj)(y)dv s (u})(z) 



() 
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where C\ is obtained from the boundedness and Lipschitz continuity of both a. 
It follows from (jTXB - (|L\3l) that 

(D.4) i^ ro p t (z>)XM) = Ej^u sup \z°(8,u,) - <( S , W )| 2 A 1 

< 2(Ct + &) J ( Ps (z°H,x») + (Df u(u))) 2 )da, 

which by Gronwall's lemma yields 

^ pt(z?(w),i?(w)) < 2(CT + C 1 )exp(2(CT + C7 1 )) jf K^)))^. 

This together with the definition of the Wasserstein metric D?-. leads to the contrac- 
tion inequality: 



(Df (MM, f (w))) 2 < 2(CT + Ci) exp (2(CT + d)) f (uj^w), 



2 



By following a similar argument as in |47j (Theorem 1.1), we can show that {A k (n(ui)) : 
k > 1} forms a Cauchy sequence a.s. in the complete metric space M? p , < (i < 1, 
and converges a.s. to a unique (a.s.) fixed point of A. □ 

Appendix E: Proof of Lemma 16.91 (i) (|5 . 10[) gives 

zo(s,u) = zo(0) + J (J fo[T,z ,u ,y]dfj, T (u))(y) S jdT + J Q [t]<1wo(t, uj), 

z'o(s,u) = z (0) + / ( / /o[T,z ,ud,;i/]d/x T (u;)(y)W + / cr [r]d«;o(r, w), 
Jo v Jr™ 7 Jo 

corresponding to the control processes uq and u' in ([0, T] x 57 x M"; {/q)- By 

the Lipschitz continuity of /o (see (A4) and (A5)) there are positive constants Co 
and Ci such that 



\z (s,u) - z' (s,uj)\ 2 < 2C s / \z (t,u}) - z' (r,uj)\ 2 dT 

Jo 

+ 2Cis 2 sup kio(t,a;,2;) — ud(t,u;,a;)| . 

(t,x)e[0,T]xR" 

The Gronwall's lemma yields 

p t (z (ll>), z' (lo)) < 2C\t 2 exp(2C i) sup \u (t,oj,x) - u' (t,u, x)\ 2 . 

This together with the fact that (if(oj) = 5 Zo {t,to) an d = $z' a (t,u)i an d the defini- 

tion of the Wasserstein metric D p , ■. leads to (|6.12[) where Co := 2C\T 2 exp(2CoT). 
(ii) We have 



Zo(s,ui) = z a (0) + ( fo[T,Zo,Uo,y]dfj, T (u))(y))dT+ a Q [T]dw (T,u), 
z' a (s,u}) = z (0) + / (/ fo[T,z l Q,Uo,y\dv T (u){y) S jdT+ I a [T]dw (T,u}), 
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corresponding to the stochastic measures H(u),v{u) <= M? p , < < 1. By the 
Lipschitz continuity of /o (see (A4) and (A5)) and Uq there are positive constants 
Co and C\ such that 



\zq(b,w) - z' (s,u)\ <2C s \z (t,w) - z' (t,u)\ dr 

Jo 

+ 2C iS 2 (d p t (h{lj) :V {lj))^ 

The Gronwall's lemma yields 

Pt(zo(u),4(u)) < 2C 1 t 2 exp(2C t)(l%(ii(u),u(u))^ 

This together with the fact that ($((*)) = 5 Za (t,ui) an d ( w ) = ^z' (t,u)t and the defini- 
tion of the Wasserstein metric Df* leads to (I6.13|) where c\ :— 2C\T 2 exp(2CoT). 
(iii) (pT20| gives 

Zi(s, u},u}') = Zi(0) + ( / f[s,z i) u,y,z]dfj l °(w)(y)dii a (w)(z))ds 

a[s,Zi,y, z]d/j, a (u) (y)d^ s (u) (z) ) dun (s, a/), 



z t '(s,w,a/) = Zi(0) + / (/ / /[s,2 l ',u',y,z]d/z°(w)(?/)d^(w)(z))(is 
( / / cr [s,z-,y,z]d^°(w)(y)ci^ s (w)(z))rfw l (s,cj'), 



corresponding to the control processes u and v! in C^p^dO, T] x ft x R n ; {/). By 
the Lipschitz continuity of / and a (see (A4) and (A5)) there are positive constants 
Co , Ci and C2 such that 



E u \ Zi (s,uj,uj') - z'^cj,^ 2 < 2(3C * + 2Ci)E (l , / |z (r,w)-z (T,w)| 2 dT 

Jo 

+ 2(3C s + 2d)E u f (dp(h{u),v(u)))' 



2 



+ 6C2S 2 sup-E 1 ^ w, x) — u'(t,u>, x)\ . 
The Gronwall's lemma yields 

ft (zi(s,w),^(fl,w)) < 2(3C i + 2C 1 )exp(2(3Co^ + 2C 1 ))^ (^(^LKw)))* 
+ 6C 2 i 2 exp (2(3C t + 2Ci)) sup \u(t,u>,x) - u'(t,uj, x)\ 2 . 

t,X 

This together with the definition of the Wasserstein metric leads to 

(D p T ([i(oj),v(oj))y < K(T) £ (dp t (^),u{w)))' ''dr 

+ K f (T) sup |u(t, cj, x) — u'(t, cj, x)| 2 , 

where if (T) := 2(3C T+2Ci) exp (2(3C T+2Ci)) and K'(T) := 6C 2 T 2 exp (2(3C T+ 
2Ci)). Applying the Gronwall's lemma gives (I57T4]) with c 2 := K'(T) exp(K(T)). 
(iv) The proof of this part closely resembles that of Part (iii) . □ 
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Appendix F: The Sensitivity Analysis of the SHJB Equations. In this 
section we study the sensitivity of the major and minor agents' SHJB equations (|5.8|) 
and (|5.18p to the stochastic measures (u) and /x9 (w) in order to show the feedback 
regularity conditions. The analysis of this section is based on the framework of Section 
6 of [25]. 

First we consider a family of stochastic optimal control problems (SOCP) (|3.ip - 
(|3.2j) parameterized by a £ 1. In this a-parameterized formulation called (SOCP) Q : 
(i) the dynamics of the states z a (t,uj), denoted by (|3.1[L . are of the form (|3. II) 
with f[t,u!,z,u], <j[t,w,z] and q[t,u>,z] replaced by f a [t, lj, z a , u a ], a a [t,ui, z a ] and 
<; a [t,w, z a ], respectively, and (ii) the cost functions J a (u a ), denoted by (13.2^ ^. are of 
the form (|3.2j) with L[t,uj, z,u] replaced by L a [t,ui, z a ,u a }. 

The value functions 4> a (-,x(-)) correspond to the (SOCP) Q are defined similar to 
(|3.3p with L[t,Lu, z,u] replaced by L a [t, w, z a , u a ]. Based on [45] we shall restrict to 
the case where </>"(•, x(-)) are semi-martingales of the form (|3.7p with r(-,a;(-)) and 
ip(-,x(-)) are replaced by T a (-,x(-)) and ip a (-, x(-)), respectively. 

If the a-parameterized family of processes <t> a {t,x), T a (t 7 x) and ip a (t,x) are a.s. 
continuous in (x,t) and are smooth enough with respect to x, then by using the 
analysis in [55] we can show that the pairs (d) a (s,x),ip a (s,x}) satisfy the follow- 
ing backward in time a-parameterized stochastic Hamilton- Jacobi-Bellman (SHJB) Q 
equations: 

(F.l) - df{t,u,x) = \H a [t,uj,x,D x (l) a (t,Lo,x)] + (<T a [t,w,x],D x ip a (t,L),x)) 

+ \tr(a a % u, x]D 2 x J a (t, w, x))] dt - (</>" ) T \t, w, x)dW(t, w), a (T, a,) = 0, 

where a a [t,uj, x] :— (J a [t,uj, x](a a [t, uj,x]) T + ^ a [t,uj, x](q a [t : uj : x]) T , and the stochas- 
tic Hamiltonians H a : [0, T] x Q x R" x R™ ->• K are given by 

H a [t, w, x,p] := inf {(/«[*, w, x, u] ,p) + L Q [t, w, s, u] } . 

Suppose the assumptions (H1)-(H3) hold for (f a ,L a , a a ,^ a ). Then the (SHJB) Q 
equations (jF.lP have unique solutions (see Theorem I3. II or Theorem 4.1 in [45]): 

(r(t,x),i, a (t,x)) G (L% t ([0,T];R),L%([0,T]-,R m )), V«el. 

The forward in time J^-adapted optimal control processes of the (SOCP) Q 
(|3~T]) »- (l3T2]) ^ are given by (see [45]) 

(F.2) u a '°(t,LU,x) := arg inf H a ' u [t, lj, x, D x (j) a {t, U3, x), u a ] 

u a eu 

= arg M v {(f a [t, u, x, u a ] , D x dp a (t, u, x)) + L a [t, lj, x, u a ] } . 

We set 

g a [t, u, x, 4> a (t, w, x), V Q (t, w, x)] := £P [t, w, x, D x cp a {t, w, as)] 

+ ( ( j tt [t, w , a; ],Z? ;c V Q (i,w,x)), 
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where A a in [0,T] x 17 x R™ is an operator on C 2 (R. n ). We may now rewrite the 
backward in time a-parameterized (SHJB)q, equations (|F.1I) as 

(F.3) d(p a {t,uj,x) + A a {t,uj,x){(j) a {t,uj,x))dt 

= -g a [t, uj, x, <j) a {t, uj, x), ip a (t, uj, x)]dt + (ip a ) T (t, oj, x)dW(t, uj), 

with <p a (T,x) = 0. 

At this point we introduce the mild form of (|F.3|) because this form is more 
suitable for the sensitivity analysis of this section. We note that it is sufficient to 
consider the mild solution in the analysis of existence and uniqueness of solutions to 
the SMF system. 

If the pair ((f> a (t,x),ip a (t,x)) is a smooth solution to (|F.3[) that satisfies the 
following mild form by a Duhamel Principle |25) : 

(F.4) 

<f> a (t, oj,x) = 1^ exp^J^ A a (r,uj,x)dr^ ^g a {s,uj,x,(f> a (s,uj,x),ip a (s,uj,x)}^ds 
exp( / A a (T,uj,x)dT S j^ a ) T (s,oj,x)' S jdW(s,uj). 

t *■* t 

We define the operators: 

$ a (t, s, oj, »)(■) = exp ( J° A a (r, uj, x)(-)dr) = exp ( J' hi(a a [r, oj, x]D 2 xx {-))dT, 

* a (t,s,uj,x)(-) = J d a A a {r,uj,x){-)dr = j ^{d a a a [T,uj,x]D 2 xx (-))dT, 

in [0,T] x ft x W 1 which are maps on C°°(M") and C 2 (R"), respectively. 
Differentiating (|F.4[) with respect to a gives 



(F.5) d a (j) a (t,uj,x) = ^ ($ Q (i,s,w,a;))(* a (t,s,w,a;)) 

(5° [s, w, a;, a (s, w, x), <if) a {s, uj, x)]jds 
+ jf ($ Q (t,s,tJ,a;))^ Q 3 Q; [s,a;,a;,(?!) a (s,cj,a;),'(/' a (s,w,a;)])rfs 

>°(f , s, w, x)) (* Q (i, s, w, x)) ((^ Q ) T (s, W, *)) dW(*, uj) 

> a (f , s, w, x)) ((cW a ) T ( s , w, «)) «), 



T 



T 



where 



d a g a [t, uj, x, 4> a (t, uj, x), ip a (t, uj, x)] = d a H a [t, uj, x, D x cf) a (t, uj, x)] 
+ d p H a [t, uj, x, D x cj> a (t, uj, x)]D x (d a ^ a {t, uj,x)) 
+ (d a a a [t, uj, x] , D x i\> a (t, uj,x)) + (a a [t, uj, x] , D x (d a <p a (t,uj,x)) ) . 



10 



M. Nourian and P. E. Caines 



We may rewrite (|F.5[) as 



(F.6) d a cf> a {t,uj,x) = J (<!> a (t,s,uj,x))A1(s,uj,x)(d a (t) a (t,uj,x))ds 

+ j (<S> a {t, s, u, x)) (b% [t, s, u, x, d a r/j a ])ds 

- ^ ($ Q (t, s, u, x)) ((0 a ^ a ) T (*, u, xj) dW(s, U ), 

- j ($ Q (t, s, u, x)) (h% [t, s, u, x])dW(s, u), 

where 

A?(a,u,x)(-) :=d p H a [s,cj,x,D x( f> a (s,u},x)}D x (-), 

h% [t, s, w, x, d a tp a ] := (* Q (t, s, ui, x)) (g a [s, u, x, c/> a (s, ui, x),ip a (s, u), x)]\ 

+ d a H a [s, w, x, D x cj) a (s, uj, x)\ + (d a <j a [s, w, x] , D x ^ a (s, w, x)) 
+ (a a [s,u;,x],D x (d a ip a )), 

hg[t,8,u,x] := {^ a (t,s,u;,x))(^p a ) T (s,0J,x)). 
We introduce the following assumption: 

(H5) d a f a [t,x,u], d a L a [t,x,u], d a a a [t,x] and d a q a [t,x] exist and are C°°(R"). 
Assume (H1)-(H3) hold where (/, L,a, ?) are replaced by (d a f a , d a L a , d a a a , d a ^ a ), 
and all the boundedness assumptions are uniformly. 

Proposition F.l. Assume (K11)-(H3) hold for (/", L a , a a , <; a ). Let the pair 
(<f) a (t,x),ip a (t,x)) be the unique solution to fF. 1\) which are C°°(R") and a.s. uni- 
formly bounded. In addition, we assume (H5 ) holds. Then, the equation \F. 51) has a 
unique solution 

(d a ^(t,x),d a i>(t,x)) e {L%([0,T};R),L%([0,T};R m )) 

such that sup 0<t<r \D x d a (p(t, -)| < oo (a.s.). 

Proof. The proof of existence and uniqueness of solution to (|F.6|) follows from 
Theorem 4.1 in [T7] (see the proof of Theorem 4.1 in [35], see also [351 [Ml US] or 
Chapter 5 of [33 ). By taking the conditional expectation Ej*»a of the square of 
both sides of (|F.6[) and the boundedness assumptions in the theorem, one can show 
sup 0<t<T \d a (j>(t, -)| < oo (a.s.) (see the proof of Theorem 2.1 in [13). Using this in 
equation (|F.6[) implies the boundedness of D x d a <f){t, ■). □ 

Appendix G: The Major and Minor (MM) SMF Linear-Qudratic-Gaussian 
(LQG) System. We consider the MM LQG dynamic game problem of [IS]. In this 
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case all functions in (I2.1[) - (|2.4I) are given by (see Remark l2.1j) 



f [t, z»(t), <(f), zf{t)] = A z»(t) + B u£(t) + F z»(t), 



f[t, z?(t), u»(t), z^(t),zf(t)} = Az?(t) + Bu?(t) + Fzf{t) + Gz?(t), 
a [t, z»(t), zf(t)] = So, a[t, af (t), z${t),zf (*)] = S, 



N 



L [t,z»(t),u»(t),z»(t)}= <(*)-(F (_£ 2 f(t)) + 



Qo 



3=1 



no 



tf(*)-(ffo(£ £*?(*)) 

7=1 

L[t,zf(t),uf(t),z^(t),zf(t)} = [z?{t)-(Hz»(t)+A(±£z?(t))+ri 

1 w 



1 T 



3 = 1 

(u?(t)) T Ru?(t), 



3 = 1 



with the deterministic constant matrices: (i) Aq, Fq, A, F,G, Ho, H and H in R nx ™, 
(ii) B and £ in R nxk , (hi) So and 5 in R" xm , (iv) the symmetric nonneeative dchnite 
matrices Qo and Q in R rax ™, (v) the symmetric positive definite matrices Rq and R 
in R fcxfc , and the deterministic constant vectors rj and no are in R™. 

In this formulation the major agent's SMF system (|5 ,8[) - (15 . 10[) is of the form 

(G.l) - dfait, oj, x) = [(Aox - -BoR^B^D^t, u, x) + F z°(t, w), D x (f>o(t, u, x)) 
+ (x- {H z°(t, u) + no), Q (x - (H z°(t, u) + m )) ) 



+ (S ,D x ijo(t,uj,x)) + -tr((S* T ' So)D 2 xx <t> (t,u,x)) 

- i>o (t,U),x)dwo(t,u), 
)-1dT, 



dt 



<po(T,x) = 0, 



(G.2) u°{t,u,x) = --Ro L B^D x 4, (t,iJ,x), 



dt 



(G.3) dz° Q \t, u) = [A z°(t, u) + B u°(t, u, z °) + F z°(t, u) 

+ S Q dw {t, u), Zq(0) = z (0), 

and the minor agents' SMF system (|5.18[) - (|5.20[) is given by 

(G.4)- dcj)(t, u, x) = {Ax - jBR- 1 B T D x <j)(t, uj, x) + Fx + Gz°(t, uj),D x cf>(t, u, x)) 
+ (x- (Hz°{t, uj) + Hx + n),Q(x- (Hz°(t, uj)+Hx + r,)) ) 
+ ^tr((S T S)D 2 xx <f){t,uj,x)) dt - i> T {t,uj,x)dw{t,oj), 4>q{T,x)=0, 



(G.5) u°{t,uj,x) = --R- 1 B T D x (j){t,uj,x), 

(G.6) dz°(t,uj)= \Az°(t,u) + Bu°{t,uj,z°) + F z°(t,uj) + Gz$(t,Lj) 



dt 



Sdw(t, lj), 



z o °(0)=z o (0). 
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Let n (-) > be the unique solution of the deterministic Riccati equation 

d t n Q (t) + n (t)A + A?n {t) - iIo&BoR^b^^) + q = o, n (r) - o. 

We denote A (-) = A — B Rq 1 Bq n (-). It can be verified that the pair (<^ , ipo)(t,u}, x) 
in (|5.8p is given by 

(f>o(t,oj,x) = x T Tlo(t)x + 2x T s (t,uj) + g (t,cv), 
ipQ(t,u,x) = 2x T q (t,uj) + ho(t,u), 

where (sq, qo)(t,uj) and (go,ho)(t,Lo) are unique solutions of the following Backward 
Stochastic Differential Equations (BSDEs): 



ds Q (t,ui) = \AS(t)s (t,w) + (n (t)F - Q H Q )z°(t,uj) - Q Q r]o\dt 

- q (t,L))dw (t,u), s (T) = 0, 
dg (t,uj) = [- sHt,u])B Q ^B^s Q {t,uj) + 2F z o (t,uj) +2 tv(S^q (t,oj)) 

+ (H a z°(t,u) + m^QoiHoz^t^) + m ) + tr(S^ n (t))" 

- ho(t,w)dw (t,u), g (T) = 



dt 



We may now express the major agent's SMF-LQG system (|G.1[) - (|G.3I) in the 
following form: 



ds (t,u) = A%(t)s (t,Lj) + (U Q (t)F Q - Q H Q )z°(t,uj) - Q m 



dt 



- q (t, u)dw (t, u), s (T) = 0, 

u° Q {t, u) = -R^B^noi^z^t, u) + s (t, u)) , 
dz°(t,uj) = fAo(t)«o(*.") - BoRo^^Uoi^soi^uj) + F z o (t,u) 



dt 



+ S a dw {t,uj), 



*8(o) - Mo), 



where z°(t, lu) is the mean field behaviour of the minor agents (see the minor agents' 
SMF-LQG system below). 

In a similar way, let II(-) > be the unique solution of the deterministic Riccati 
equation 



d t Tl(t) + U(t)A + A 1 n(t) - TL(t)BR B IL(t) + Q = 0, U(T) = 0. 

We denote A(-) = A — BR" 1 B T Ii{-). It can be verified that the pair (4>, i/j)(t, lo, x) in 
(I5.18[) is given by 



cf>(t,u),x) = x T U(t)x + 2x T s(t : oj) + g(t,w), 
ip T (t,uj,x) = 2x T q(t,uj) + h(t,w), 
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where (s,q)(t,uj) and (g,h)(t,uS) are unique solutions of the following BSDEs: 

- ds(t,u) = \A T (t)s(t,uj) + (U(t)F - QH)z°(t,uj) + (n(i)G - QH)z°(t,uj) 
— Qrj dt — q(t,uj)dwo(t,uj), s(T) = 0, 

- s T (t, uj)BR- 1 B T s(t, u) + 2Fz°(t, tj) + 2Gz%(t, u>) 
+ (Hz°(t,u) + Hz°(t,u)+r 1 ) T Qo(Hz (t,u J )+Hz°(t,uj)+r 1 ) 
+ tr(S T SIL(t)) 1 dt - h(t, u)dw {t, uj), g{T) = 0. 



dg(t,u) 



We may now express the minor agents' SMF-LQG system (|G.4[) - (|G.6I) in the 
following form: 

-ds(t,w) = \A T (t)s(t,uj) + (n(i)F - QH)z°(t,u) + (U(t)G - QH)z£(t,u) 

— Qrj dt — q(t,ui)dwo(t,uj), s(T) = 0, 

u°(t,u) = -R- 1 B T (n(t)z°(t,u}) + s(t,w)), 

dz°(t,ui) = \(A(t)+F)z°(t,uj) - BR- x B T Il(t)s(t,u}) + Gzg(t,u))]dt 

+ Sdw{t,uj), z°(0) = z(0). 

So we retrieve the SMF system for the MM LQG dynamic games model of [37] 
for minor agents with uniform parameters (see equations (10)-(11) and (22)-(23) in 
|37j . see also [IB]). The reader is referred to [37] for an explicit representation of a 
solution to the SMF-LQG system under some appropriate conditions. 

Appendix H: A Nonlinear Example. In this section we present a major and 
minor version of the synchronization of coupled nonlinear oscillators game model |51] . 
Consider a population of N + 1 oscillators with dynamics 



(H.1) 



d6f(t) = uf(t)dt + adwj(t) (mod 2tt) 



< j < N, t>0, 



where 0j(t) € [0, 2tt] is the phase of the j th oscillator at time t, Uj(-) is the control 
input, a is a non-negative scalar, and {wj : < j < N} denotes a sequence of 
independent standard scalar Wiener processes (see [51]). It is assumed that the initial 
states {0j(0)} are chosen independently on [0, 2ir]. The objective of the j th oscillator 
is to minimize its own cost function 



(H.2) 
(H.3) 



J N (u»,u» ) :=E [ (^E sin2 Kit) - 6?(t)] +r(u»(t)) 2 )dt, 
Ja k=i 

T N 

J? (uf ,u^) :=E (A E sin2 - ( X6 o(t) + (1 - A)^W) 



fc=i 



+ r(u?(t)) 2 )dt, 



l<i<N, 



where r is a positive scalar and A € (0, 1). 
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Similar arguments in previous section yield the following major agent's SMF 
system (|577|i and (pTSl) - ([?!?]) : 

-d<f>o(t,u>,x) = - ^-(d x (j)o(t,uj,x)) 2 +mo(t,u,x) + ad x tpo(t, uj, x) 
L 4r 

+ -^-9l x (j>o{t,u,x) dt-ip (t,u),x)dw (t,u), (j) O (T,x) = 0, 



ul(t,w,x) 



2r 



d x <po(t,u,x), 



dp a s (t,uj,x) = [ia x ((fl !e ^o(t ) w,x))p2(t,u,a ; )) +yy(t,u,i) 



mo(t, Wj x) 



ad x p° s {t,uj,x)dwo{t,uj), 
sm 2 (x - d)p{t,uj,e)d6, 



p° s (s,x) = 5go (s) (dx), 



where mo(t,w,a:) is called the infinite population cost-coupling of the major agent, 
and Oq(-) is the solution of the closed-loop equation 



by 



d6° a {t) = ul(t,6° a (t))dt + adw Q (t) (mod 2tt) t > 0. 

In a similar way, the minor agents' SMF system (|5.17|) and (|5.18[) - (|5.19[) is given 



d(f>{t, uj, x) 



- (dx<t>(t, uj, x)) 2 + m(t, u>, x) + %rd' 1 x J>{t, ui, x) 
ip(t, uj, x)dw(t, uj), 



dt 

4>(T,x)=0, 



u°(t, uj, x) = -—d x <j>(t, uj, x), 
2r 



dp{t,uj,x)= —d x ( K [d x (j){t,uj,x))p{t,uj,x) S j+^-dl x p{t,uj,x)dt, p(0, 



m(t, uj, x) 



2-7T /.27T 



JO 



sin 2 (x - (X6 + (1 - \)0))po(t, u, )p(t, uj, 9)d9 d8, 



where m(t, uj, x) is called the infinite population cost-coupling of the major agent. The 
reader is referred to the deterministic mean field system (14a)- (14c) in [51] for the 
synchronization of coupled nonlinear oscillators game model with only minor agents. 



